José David ArévaloinAWS in Plain EnglishFrom Redshift to Athena: A Data Engineering Team’s Journey to 90% Cost Reduction and Performance…As a data engineer, I’ve worked on numerous projects where balancing performance with cost has been a critical challenge. Today, I’m…Sep 244Sep 244
José David ArévaloinAWS in Plain EnglishWhen AWS Athena costs skyrocket: Key lessons and how to avoid themIn the realm of data engineering, choosing the right tools and technologies is critical for efficiently managing and analyzing data. AWS…Jul 3Jul 3
José David ArévaloinPython in Plain EnglishIntroducing AtomicExecutionControl: Streamline Your Data Engineering WorkflowsIn the rapidly evolving data engineering landscape, the introduction of serverless architectures like AWS Lambda and Fargate has unlocked…Mar 20Mar 20
José David ArévaloinAWS in Plain EnglishA Serverless, User-Centric Framework: Automating Athena Query ExecutionIn today’s dynamic big data environment, speed and efficiency in query execution are paramount. AWS Athena serves as a potent tool for…Dec 28, 2023Dec 28, 2023
José David ArévaloFrom grieving to opportunity: The journey of the survivors of layoffsIn a scenario where companies of all sizes, from Google or Amazon to startups, announce layoffs, those of us who remain in our positions…Sep 22, 20231Sep 22, 20231
José David ArévaloDel duelo a la oportunidad: La travesía de los sobrevivientes a despidos masivosEn un escenario donde empresas de todos tamaños, desde Google o Amazon hasta startups, anuncian despidos masivos, quienes permanecemos en…Sep 22, 2023Sep 22, 2023
José David ArévaloinAWS in Plain EnglishServing Data Safely: Managing Versions and Environments with API Gateway and LambdaAs a data engineer, you might not first think of creating and maintaining APIs; instead, you’re likely more involved in data pipelines, ETL…Sep 1, 2023Sep 1, 2023
José David ArévaloinAWS in Plain EnglishScalable Data Processing with AWS Serverless Scatter-Gather Pattern ImplementationData engineering plays a critical role in managing, processing, and extracting value from substantial volumes of data. The Scatter-Gather…Apr 21, 2023Apr 21, 2023
José David ArévaloinAWS in Plain EnglishSlash Command in Slack to trigger Athena queries and receive reportsIn our daily work as data engineers, we need to design and implement pipelines to be the data accessible to the users, no matter if the…Aug 26, 2022Aug 26, 2022
José David ArévaloinLevel Up CodingCreating and filling a Postgres DB with Docker composeMany times we need to be able to fill and eventually share a database with dummy data, either to test our pipelines, test queries, make a…Jan 30, 20216Jan 30, 20216