List: Data Engineering | Curated by Giorgos Myrianthous

Aug 11, 2024
66 stories
1 save
Data EngineeringMy articles about Data Engineering
In
Making Plum 🛠️
by
Giorgos Myrianthous
dbt + Airflow = ❤Building dbt-airflow: A Python package that integrates dbt and Airflow
Apr 16, 2024
11
Apr 16, 2024
11
In
TDS Archive
by
Giorgos Myrianthous
Modeling Slowly Changing DimensionsA deep dive into the various SCD types and how they can be implemented in Data Warehouses
May 3, 2024
1
May 3, 2024
1
In
TDS Archive
by
Giorgos Myrianthous
Deploying dbt Projects at Scale on Google CloudContainerising and running dbt projects with Artifact Registry, Cloud Composer, GitHub Actions and dbt-airflow
Jul 29, 2024
2
Jul 29, 2024
2
In
TDS Archive
by
Giorgos Myrianthous
Mastering Airflow VariablesThe way you retrieve variables from Airflow can impact the performance of your DAGs
Jan 27, 2024
4
Jan 27, 2024
4
In
Level Up Coding
by
Giorgos Myrianthous
Creating Airflow Local Development Environment with Astro CLIGetting Airflow up and running in less than a minute
Dec 28, 2023
1
Dec 28, 2023
1
In
TDS Archive
by
Giorgos Myrianthous
Running Airflow DAG Only If Another DAG Is SuccessfulUsing Airflow sensors to control the execution of DAGs on a different schedule
Dec 19, 2023
1
Dec 19, 2023
1
In
TDS Archive
by
Giorgos Myrianthous
Transitioning from ETL to ELTHow cloud computing and analytics engineering forced the transition from ETL to ELT
Dec 6, 2023
2
Dec 6, 2023
2
In
TDS Archive
by
Giorgos Myrianthous
An Introduction To Analytics EngineeringWho is an Analytics Engineer and what are they supposed to do
Oct 22, 2023
3
Oct 22, 2023
3
In
TDS Archive
by
Giorgos Myrianthous
ETL vs ELT vs Streaming ETLExploring batch and real-time design paradigms for data processing
Aug 29, 2023
1
Aug 29, 2023
1
In
TDS Archive
by
Giorgos Myrianthous
Airflow 2.7 Is Now OutHere are the most important feature updates that will make your life easier and save you time
Aug 22, 2023
2
Aug 22, 2023
2
In
Level Up Coding
by
Giorgos Myrianthous
How To Run dbt on AirflowAutomated rendering of dbt projects on Apache Airflow
Aug 3, 2023
2
Aug 3, 2023
2
In
TDS Archive
by
Giorgos Myrianthous
Processing Data At Scale With MapReduceA deep dive into MapReduce and parallelization
Jul 19, 2023
Jul 19, 2023
In
TDS Archive
by
Giorgos Myrianthous
What Are ACID Transactions?Understanding ACID properties in the context of database transactions
Jul 5, 2023
1
Jul 5, 2023
1
In
TDS Archive
by
Giorgos Myrianthous
How To List All BigQuery Datasets and Tables with PythonProgrammatically list all datasets and tables using BigQuery API and Python
May 15, 2023
May 15, 2023
In
Level Up Coding
by
Giorgos Myrianthous
Fixing Error: Runtime Error could not find profile in dbtHow to provide a connection profile when running dbt commands
May 14, 2023
1
May 14, 2023
1
In
TDS Archive
by
Giorgos Myrianthous
Running Jaffle Shop dbt Project in DockerA containerised version of the popular Jaffle Shop dbt project
Apr 28, 2023
Apr 28, 2023
In
Geek Culture
by
Giorgos Myrianthous
9 Docker Commands For Data Scientists and EngineersThe most commonly used Docker commands for day-to-day tasks
Apr 8, 2023
1
Apr 8, 2023
1
In
TDS Archive
by
Giorgos Myrianthous
How to Set GOOGLE_APPLICATION_CREDENTIALS in PythonConfiguring Application Default Credentials and fixing oauth2client.client.ApplicationDefaultCredentialsError
Jan 19, 2023
2
Jan 19, 2023
2
 This story is no longer available
In
TDS Archive
by
Giorgos Myrianthous
How To Fix TypeError: ObjectId Is Not JSON SerializableConverting a mongo cursor into JSON object in Python
Jan 7, 2023
1
Jan 7, 2023
1