Using Kafka to solve food wastage for millions.

The food wastage in India is 70 tonnes per year, and there is mismanagement at several layers. Approximately 20-30% of the wastage happens in the last mile, between wholesale traders, and retail mom-and-pop stores. Is there something we can do about it? This was the problem statement I attempted to solve as a first engineering hire at a startup. Our customers were 12.8 million retail owners that deal in FMCG (Fast-moving consumer goods, such as food grains, tooth paste, etc.). The goal was to develop a platform for retail traders (mom and pop shop owners / small and medium business owners) to buy FMCG products from wholesale traders using an Android app. We used cloud extensively to develop micro-services for a section of the population which is not very well versed with smartphones and technology. In this domain, data is very unstructured and we had to come up with challenging ways to representing this unstructured data for a slick browse and search experience. We used Kafka as the event streaming platform in conjunction with DynamoDB, Elastic Search, and Redshift to build pipelines and systems on Cloud. Data ingested by sellers in DynamoDB would be transformed and published in ElasticSearch & inserted into Redshift tables for analysis. I'd like to delve into these details as part of this session.

Tejas Chopra is a Senior Software Engineer, working in the Data Storage Platform team at Netflix, where he is responsible for architecting storage solutions to support Netflix Studios and Netflix Streaming Platform. Prior to Netflix, Tejas was working on designing and implementing the storage infrastructure at Box, Inc. to support a cloud content management platform that scales to petabytes of storage & millions of users. Tejas has worked on distributed file systems & backend architectures, both in on-premise and cloud environments as part of several startups in his career. Tejas is an International Keynote Speaker and periodically conducts seminars on Software Development & Cloud Computing and has a Masters Degree in Electrical & Computer Engineering from Carnegie Mellon University, with a specialization in Computer Systems.

picture of Tejas Chopra

Tejas Chopra

Senior Software Engineer, Netflix