01First, check source access
The source repository remains private. The link below is for authorized collaborators and may return 404 when you are not signed in or lack permission. This site does not provide a public source download or promise unrestricted clone access.
If you already have access, clone the repository or open your existing checkout. Run all commands below from its root. If you do not, use the public lessons and upstream references; local reproduction becomes available after the owner grants access.
02Prepare the environment
- Use Bash on macOS or Linux. On Windows, use a Linux shell inside WSL2 with Docker integration; native PowerShell is not the launcher environment.
- Install Docker with Compose v2, Python 3 and curl. A working Docker daemon is required. Colima is optional on macOS, not a dependency on every platform.
- Plan for at least 10 GiB available to Docker and around 30 GiB of free disk for images and data. These are planning baselines, not validated minimums or performance claims.
- The first build downloads container images and connector JARs. Allow network access to their registries. Image availability and architecture support must be checked on your machine.
- The launcher supports these platform conventions; this edition does not claim a fresh full-stack integration run on every supported platform.
03Initialize, diagnose, start
Initialization creates local configuration without replacing an existing file. Review the example and your port choices, then run the doctor. Resolve errors before launching the stack.
Open starts the local services and prints the console address. Use that address from the same machine. Keep these learning services on loopback; the public reading site never connects to them.
Start your Docker environment first. If you choose Colima on macOS, ./stream-lab open --colima starts it explicitly. Normal close leaves the shared Docker VM running.
./stream-lab init
./stream-lab doctor
./stream-lab open
./stream-lab status
./stream-lab smoke04Use one clear operating loop
| Task | Command | What it means |
|---|---|---|
| Inspect readiness | ./stream-lab status | Show the local project’s container state |
| Check the data path | ./stream-lab smoke | Run service, job and query checks; inspect any failed assertion |
| Read logs | ./stream-lab logs | Follow recent service logs; stop following with Ctrl-C |
| Run bounded calculations | ./stream-lab batch | Execute hourly aggregation and count audit |
| Open advanced SQL | make sql | Use the local Flink SQL client for independent queries |
| Stop the lab | ./stream-lab close | Stop this project and retain named volumes |
05Troubleshoot from the first failing boundary
| Symptom | Next check |
|---|---|
| Docker unreachable | Start your chosen Docker environment and rerun doctor. |
| Port already allocated | Check the local configuration and existing listeners; choose unused ports before retrying. |
| Image or connector download fails | Inspect the first build error and registry access; confirm image architecture compatibility. |
| Containers run but no data arrives | Check MySQL and mock-producer, then CDC job state, then the join job. |
| Lake query times out | Check SQL Gateway, available Flink slots, tiering and sink commit errors. |
| Batch summary appears old | Check batch-runner logs and job completion, then Kafka consumption. |
| After editing SQL, jobs duplicate | Cancel the old job deliberately before resubmitting. Do not repeatedly launch duplicate continuous jobs. |
06Know what stopping and retention preserve
Stopping the lab preserves named volumes. It does not make a backup, expire all old lake files or establish a durable high-availability recovery plan. Do not use volume-deletion commands unless you intend to erase your experiment data.
The code has seven-day settings for several source, log, state and serving layers. They have different semantics and cleanup schedules. Iceberg snapshot expiration and comprehensive lake-file cleanup are not implemented as a scheduled retention workflow.
Local components use learning-only authentication arrangements. Do not tunnel their ports or publish the Compose stack as a public service. Configuration and raw local logs are not part of the website export.