Worked in a backend team that handled over 1 billion events a day across many of our services. Many of the services exposed functionalities via REST API and they were written using Java/ Golang/ Python. Infrastructure required by our services was provisioned, managed and monitored by us (Terraform, ELK, Prom, Grafana).

• Designed and implemented a Kafka Streams service written in Java that processed and validated 1 billion documents daily, improving delivery quality to end-customers.
• Developed a Spring Boot based REST API that acted as a gateway between the application’s website and different providers across the company. Leading to deprecation of legacy components and thus lower costs and ease of maintenance.
• Built scalable data streaming services using Python and Go, which extracted data from social media providers such as Facebook, YouTube, and Twitter. The data constituted approximately 20% of the data used for the company’s end-product.

Worked in a data platform team responsible of managing and provisioning data related services. Additionally, automated data ingestion processes and improved data quality standards.

• Provisioned and managed infrastructure using Terraform and Kubernetes for the data platform team, enabling efficient development across multiple data functions.
• Designed, developed, tested and monitored over 50 Airflow based ELT (extract, load and transform) pipelines written in Python/Go. The pipelines fueled business critical functionalities with data on a daily basis.
• Worked on a company wide consolidated billing project that enabled monitoring all costs incurred by AWS based services. This improved cost awareness and led to the reduction of total AWS costs by 20%.

Worked in a Data team which was responsible ingesting and monitoring data delivery for all company internal functions (e.g. marketing, SEO, HR, finance). Additionally, we provided analytics related dashboards and reporting for the entire company.

• Led and completed a full migration from AWS Redshift to Snowflake and the relevant data pipelines/data ingestion services. This resulted in a significant improvement in general query and analytics dashboards performance.
• Automation of data ingestion and general operations by using Apache Airflow and Python based pipelines which allowed jobs to be executed on higher frequency and improved data freshness.
• Drove a project that led to the containerization of the data team’s entire setup which resulted in improved portability, consistency and scalability of our services and applications.