Aman Soni

Data + AI @ Databricks | GTM | Data Engineering | Spark, SQL, Gen AI, Azure Databricks

Databricksex-ConcertAIex-Siemens Healthcare

“

As a seasoned Data Engineer with over 5 years of experience, I excel in crafting robust data pipelines and platforms to drive analytics and data-driven insights.

More

My expertise spans across managing and processing vast datasets, optimizing system performance, and implementing cost-effective solutions. Throughout my career, I have successfully executed ETL processes handling over 200+ TB of data, significantly reducing latency, enhancing storage, and minimizing error I have engineered a robust big data framework processing, enhancing data analysis capabilities and providing analysis-ready data from 30+ sources for diverse pharmaceutical clients. My initiatives in production deployments and optimizing runtime achieved a 50% reduction, while automation of data tasks using AWS Lambda led to a 60% improvement in operational efficiency, cutting data processing time by 50 hours per Have led a team in agile-driven enterprise implementations, I have streamlined analytics and reporting processes, boosting data accessibility by 35%. My expertise in dimensional and multi-dimensional modeling, coupled with automating DQ Checks and report generation, resulted in $500K savings and reduced operational overhead by I am passionate about leveraging data to uncover insights and drive strategic decisions. I thrive in dynamic environments, constantly seeking innovative solutions to complex data challenges. Let's connect to explore how I can contribute to your organization's success with my data engineering expertise. Skills:- Programming Languages: SQL, Python, PySpark, Spark, Spark SQL- Cloud Platforms: AWS, GCP, Azure Data Lake- Scripting Languages: Bash, C++, Python, HTML- Data Warehousing: Snowflake, Amazon Redshift, Presto, Elasticsearch- ETL/ELT Tools: Microsoft Excel, Jira, Confluence, Azure Ops, Git, Gitlab, Bitbucket, Airflow- Competencies: ETL/ELT Development, CI/CD, Process Automation, Data Modeling, Business Analytics, Data Warehousing, Data Analytics, Data Reconciliation, Data Normalization/De-normalization, Data Validation, Data Management, and TestingFeel free to connect with me or message me to discuss potential opportunities or collaborations. I look forward to expanding my network and contributing to data-driven initiatives.