feat(docker): Build containers for Jaffle shop project

This commit is contained in:
George Myrianthous
2023-03-02 16:12:02 +00:00
parent a50f07742e
commit 3a6b8e315b
4 changed files with 152 additions and 0 deletions

15
Dockerfile Normal file
View File

@@ -0,0 +1,15 @@
FROM --platform=linux/amd64 python:3.10-slim-buster
RUN apt-get update \
&& apt-get install -y --no-install-recommends
WORKDIR /usr/src/dbt
# Install the dbt Postgres adapter. This step will also install dbt-core
RUN pip install --upgrade pip
RUN pip install dbt-postgres==1.2.0
RUN pip install pytz
# Install dbt dependencies (as specified in packages.yml file)
# Build seeds, models and snapshots (and run tests wherever applicable)
CMD dbt deps && dbt build --profiles-dir ./profiles && sleep infinity

100
README.md
View File

@@ -4,3 +4,103 @@ This is a containerised version of the popular
You can use this project to populate and run Jaffle Shop in a Postgres instance. You can use this project to populate and run Jaffle Shop in a Postgres instance.
# Running the project with Docker # Running the project with Docker
You'll first need to build and run the services via Docker (as defined in `docker-compose.yml`):
```bash
$ docker-compose build
$ docker-compose up
```
The commands above will run a Postgres instance and then build the dbt resources of Jaffle Shop as specified in the
repository. These containers will remain up and running so that you can:
- Query the Postgres database and the tables created out of dbt models
- Run further dbt commands via dbt CLI
## Building additional or modified data models
Once the containers are up and running, you can still make any modifications in the existing dbt project
and re-run any command to serve the purpose of the modifications.
In order to build your data models, you first need to access the container.
To do so, we infer the container id for `dbt` running container:
```bash
docker ps
```
Then enter the running container:
```bash
docker exec -it <container-id> /bin/bash
```
And finally:
```bash
# Install dbt deps (might not required as long as you have no -or empty- `dbt_packages.yml` file)
dbt deps
# Build seeds
dbt seeds --profiles-dir profiles
# Build data models
dbt run --profiles-dir profiles
# Build snapshots
dbt snapshot --profiles-dir profiles
# Run tests
dbt test --profiles-dir profiles
```
Alternatively, you can run everything in just a single command:
```bash
dbt build --profiles-dir profiles
```
## Querying Jaffle Shop data models on Postgres
In order to query and verify the seeds, models and snapshots created in the dummy dbt project, simply follow the
steps below.
Find the container id of the postgres service:
```commandline
docker ps
```
Then run
```commandline
docker exec -it <container-id> /bin/bash
```
We will then use `psql`, a terminal-based interface for PostgreSQL that allows us to query the database:
```commandline
psql -U postgres
```
You can list tables and views as shown below:
```bash
postgres=# \dt
List of relations
Schema | Name | Type | Owner
--------+---------------+-------+----------
public | customers | table | postgres
public | orders | table | postgres
public | raw_customers | table | postgres
public | raw_orders | table | postgres
public | raw_payments | table | postgres
(5 rows)
postgres=# \dv
List of relations
Schema | Name | Type | Owner
--------+---------------+------+----------
public | stg_customers | view | postgres
public | stg_orders | view | postgres
public | stg_payments | view | postgres
(3 rows)
```
Now you can query the tables constructed form the seeds, models and snapshots defined in the dbt project:
```sql
SELEC * FROM <table_or_view_name>;
```

25
docker-compose.yml Normal file
View File

@@ -0,0 +1,25 @@
version: "3.9"
services:
postgres:
container_name: postgres
image: postgres:15.2-alpine
environment:
- POSTGRES_USER=postgres
- POSTGRES_PASSWORD=postgres
ports:
- 5432
healthcheck:
test: ["CMD-SHELL", "pg_isready -U postgres"]
interval: 5s
timeout: 5s
retries: 5
dbt:
container_name: dbt
build: .
image: dbt-jaffle-shop
volumes:
- ./:/usr/src/dbt
depends_on:
postgres:
condition: service_healthy

12
profiles/profiles.yml Normal file
View File

@@ -0,0 +1,12 @@
jaffle_shop:
target: dev
outputs:
dev:
type: postgres
host: postgres
user: postgres
password: postgres
port: 5432
dbname: postgres
schema: public
threads: 1