feat(docker): Build containers for Jaffle shop project
This commit is contained in:
15
Dockerfile
Normal file
15
Dockerfile
Normal file
@@ -0,0 +1,15 @@
|
||||
FROM --platform=linux/amd64 python:3.10-slim-buster
|
||||
|
||||
RUN apt-get update \
|
||||
&& apt-get install -y --no-install-recommends
|
||||
|
||||
WORKDIR /usr/src/dbt
|
||||
|
||||
# Install the dbt Postgres adapter. This step will also install dbt-core
|
||||
RUN pip install --upgrade pip
|
||||
RUN pip install dbt-postgres==1.2.0
|
||||
RUN pip install pytz
|
||||
|
||||
# Install dbt dependencies (as specified in packages.yml file)
|
||||
# Build seeds, models and snapshots (and run tests wherever applicable)
|
||||
CMD dbt deps && dbt build --profiles-dir ./profiles && sleep infinity
|
||||
100
README.md
100
README.md
@@ -4,3 +4,103 @@ This is a containerised version of the popular
|
||||
You can use this project to populate and run Jaffle Shop in a Postgres instance.
|
||||
|
||||
# Running the project with Docker
|
||||
You'll first need to build and run the services via Docker (as defined in `docker-compose.yml`):
|
||||
```bash
|
||||
$ docker-compose build
|
||||
$ docker-compose up
|
||||
```
|
||||
|
||||
The commands above will run a Postgres instance and then build the dbt resources of Jaffle Shop as specified in the
|
||||
repository. These containers will remain up and running so that you can:
|
||||
- Query the Postgres database and the tables created out of dbt models
|
||||
- Run further dbt commands via dbt CLI
|
||||
|
||||
|
||||
## Building additional or modified data models
|
||||
Once the containers are up and running, you can still make any modifications in the existing dbt project
|
||||
and re-run any command to serve the purpose of the modifications.
|
||||
|
||||
In order to build your data models, you first need to access the container.
|
||||
|
||||
To do so, we infer the container id for `dbt` running container:
|
||||
```bash
|
||||
docker ps
|
||||
```
|
||||
|
||||
Then enter the running container:
|
||||
```bash
|
||||
docker exec -it <container-id> /bin/bash
|
||||
```
|
||||
|
||||
And finally:
|
||||
|
||||
```bash
|
||||
# Install dbt deps (might not required as long as you have no -or empty- `dbt_packages.yml` file)
|
||||
dbt deps
|
||||
|
||||
# Build seeds
|
||||
dbt seeds --profiles-dir profiles
|
||||
|
||||
# Build data models
|
||||
dbt run --profiles-dir profiles
|
||||
|
||||
# Build snapshots
|
||||
dbt snapshot --profiles-dir profiles
|
||||
|
||||
# Run tests
|
||||
dbt test --profiles-dir profiles
|
||||
```
|
||||
|
||||
Alternatively, you can run everything in just a single command:
|
||||
|
||||
```bash
|
||||
dbt build --profiles-dir profiles
|
||||
```
|
||||
|
||||
## Querying Jaffle Shop data models on Postgres
|
||||
In order to query and verify the seeds, models and snapshots created in the dummy dbt project, simply follow the
|
||||
steps below.
|
||||
|
||||
Find the container id of the postgres service:
|
||||
```commandline
|
||||
docker ps
|
||||
```
|
||||
|
||||
Then run
|
||||
```commandline
|
||||
docker exec -it <container-id> /bin/bash
|
||||
```
|
||||
|
||||
We will then use `psql`, a terminal-based interface for PostgreSQL that allows us to query the database:
|
||||
```commandline
|
||||
psql -U postgres
|
||||
```
|
||||
|
||||
You can list tables and views as shown below:
|
||||
```bash
|
||||
postgres=# \dt
|
||||
List of relations
|
||||
Schema | Name | Type | Owner
|
||||
--------+---------------+-------+----------
|
||||
public | customers | table | postgres
|
||||
public | orders | table | postgres
|
||||
public | raw_customers | table | postgres
|
||||
public | raw_orders | table | postgres
|
||||
public | raw_payments | table | postgres
|
||||
(5 rows)
|
||||
|
||||
postgres=# \dv
|
||||
List of relations
|
||||
Schema | Name | Type | Owner
|
||||
--------+---------------+------+----------
|
||||
public | stg_customers | view | postgres
|
||||
public | stg_orders | view | postgres
|
||||
public | stg_payments | view | postgres
|
||||
(3 rows)
|
||||
|
||||
```
|
||||
|
||||
Now you can query the tables constructed form the seeds, models and snapshots defined in the dbt project:
|
||||
```sql
|
||||
SELEC * FROM <table_or_view_name>;
|
||||
```
|
||||
|
||||
25
docker-compose.yml
Normal file
25
docker-compose.yml
Normal file
@@ -0,0 +1,25 @@
|
||||
version: "3.9"
|
||||
|
||||
services:
|
||||
postgres:
|
||||
container_name: postgres
|
||||
image: postgres:15.2-alpine
|
||||
environment:
|
||||
- POSTGRES_USER=postgres
|
||||
- POSTGRES_PASSWORD=postgres
|
||||
ports:
|
||||
- 5432
|
||||
healthcheck:
|
||||
test: ["CMD-SHELL", "pg_isready -U postgres"]
|
||||
interval: 5s
|
||||
timeout: 5s
|
||||
retries: 5
|
||||
dbt:
|
||||
container_name: dbt
|
||||
build: .
|
||||
image: dbt-jaffle-shop
|
||||
volumes:
|
||||
- ./:/usr/src/dbt
|
||||
depends_on:
|
||||
postgres:
|
||||
condition: service_healthy
|
||||
12
profiles/profiles.yml
Normal file
12
profiles/profiles.yml
Normal file
@@ -0,0 +1,12 @@
|
||||
jaffle_shop:
|
||||
target: dev
|
||||
outputs:
|
||||
dev:
|
||||
type: postgres
|
||||
host: postgres
|
||||
user: postgres
|
||||
password: postgres
|
||||
port: 5432
|
||||
dbname: postgres
|
||||
schema: public
|
||||
threads: 1
|
||||
Reference in New Issue
Block a user