Section 04
Data Architecture
The engine room of the Vision Statement — a medallion lakehouse on AWS, serverless-first, with consent and data-residency controls built in from day one.
Which stack is actually committed — the reconciliation
MVP: S3 · Glue Data Catalog · Athena · dbt Core · QuickSight
Run Cost
A$46.43/mo
CalculatedMonthly-cancellable, from the vendors’ own published Sydney prices.
Setup
8.0 days
7.0 of them committable before the first-party data is disclosed.
Day-1 Roster
One fractional analyst
No data-engineering hires priced — not “2–3 data engineers”.
The working — line by line
| Line Item | Assumed Volume | Published Rate | Monthly |
|---|---|---|---|
| S3 storage | 5 GB stored | US$0.025/GB-month | US$0.125 |
| Athena | 10 GB scanned per month | US$5.00/TB | US$0.05 |
| QuickSight | 1 author + 3 board readers | US$24/mo per author · US$3/mo per reader | US$33.00 |
| Glue Data Catalog | — | Published free tier | A$0 |
| dbt Core | — | Open source | A$0 |
| Total | US$33.175/mo ÷ 0.7145 (RBA rate, 21 August 2026) | A$46.43/mo |
The four rates are the vendors’ own published Sydney prices, applied to the assumed volumes shown. Metabase OSS self-hosted stands as the A$0 BI alternative — “Free unlimited users” on its own pricing page.
Growth layers return only on a measured trigger
The full 10-layer lakehouse in the graph below is post-G2, trigger-gated growth design — each heavier layer may be re-proposed only against a trigger above.
Interactive · End-to-End Architecture
From Source Systems to Marketing Activation
Every box opens in one plain sentence — and Follow one ticket traces a single sample transaction along the whole path.
Hover any node for a plain-language explainer. Click to pin it — pin two to compare them side by side.
Source Systems
Ingestion
Governed Lakehouse
S3 + Iceberg · Glue + dbt · Athena / Redshift Serverless
Data Marts
Activation
These four controls are not a stage — they run underneath every node above, and each one is a condition the stage has to satisfy before data moves on.
First-Party Platform DB: The ticketing app’s own database — the single place where a sale actually happens and the only system that knows a real buyer bought a real seat. Payments & Settlement: The payment provider’s record of what money moved — authorisations, refunds and the payouts that eventually land in the bank. Marketing Platforms: The advertising and messaging tools — what was sent, to whom, what it cost and which ticket sale the platform claims it caused. Public & Government Data: Official statistics — the census and survey releases that supply the denominators no company can produce for itself. Partner & Promoter Feeds: What the people who actually own the shows send in — venue capacities, performance dates, allocations and settlement terms. Batch ETL: The scheduled collection round — once a night (or once an hour) it fetches whole files from each source and brings them in together. Event Streaming: The live wire — transactions and app events arrive the second they happen instead of waiting for the nightly run. Landing Zone: A locked room where every incoming file is kept exactly as it arrived — encrypted, timestamped and untouched. Bronze — Raw: The permanent raw record: every field the source sent, kept in full, so the business can change its mind later and recompute from scratch. Silver — Validated: Where messy raw data becomes trustworthy data — same facts, made consistent, de-duplicated and legally usable. Gold — Business-Ready: The one version of each number the business argues from — revenue, refund rate, contribution per ticket — defined once and computed the same way every time. Finance & Unit Economics: The money view: what was sold, what was refunded, what tax is owed and what actually settled. Customer & Consent: Who the buyers are, what they have agreed to, and what the business is therefore allowed to send them. Events, Venues & Inventory: What is on sale, where, how much of it is left, and how much of the room actually filled. Marketing & Growth: What it costs to win a buyer, and whether the campaign actually caused the sale. Markets, Partners & Risk: The board’s view: which market to enter next, who the counterparty is, and what is currently going wrong. Campaign Activation: Turning a governed segment into an actual audience the marketing tools can send to. Dashboards & BI: The screens Leadership actually looks at — every tile carrying the file it came from and the label of what kind of data it is. APIs & ML Models: Machine-to-machine delivery — governed data feeds for partners, and models that price or predict. Consent & Privacy Enforcement: The control that decides, for every row, whether the business is legally allowed to use it for this purpose. Data-Quality Gates: Automated tests that stop a bad number reaching a dashboard, instead of someone spotting it afterwards. Lineage & Traceability: The paper trail: for any number on any screen, which file it came from and when that file was captured. Residency & Transfer Controls: The rules about where data is allowed to physically sit, and what has to be signed before it crosses a border.
The post-trigger target design — the committed MVP build is the five-component stack costed above.
Consumption
Five Data Marts
Finance & Unit Economics
Reconciled financial truth for revenue recognition, settlement tracking and unit-economics analysis
Key metrics: GTV, Recognised Revenue, Take Rate, Refund Rate, Settlement Variance, Contribution per Ticket
Access: CFO, Finance team, External auditors (read-only)
Refresh: Daily reconciliation; real-time for settlement exceptions
Sensitive: Payment tokens, tax fields (restricted)
Customer & Consent
Understand buyer behaviour under strict consent and privacy controls
Key metrics: Active Buyers, Repeat Rate, Consented Reachable Audience, Deletion SLA Compliance
Access: Customer team, Marketing (consented attributes only), Privacy officer
Refresh: Event-driven for consent; daily for behavioural aggregates
Sensitive: Name, email, phone, location (tokenised)
Events, Venues & Inventory
Track event supply, venue utilisation and operational performance
Key metrics: Sell-Through Rate, Venue Capacity Utilisation, Lead Time to Purchase, Scan Rate
Access: Operations team, Event managers, Promoter portal (filtered)
Refresh: Real-time inventory; daily performance metrics
Sensitive: Operational access tokens (restricted)
Marketing & Growth
Measure acquisition efficiency and campaign performance
Key metrics: CAC, ROAS, Conversion Rate, Incrementality
Access: Marketing team, Growth team, CMO
Refresh: Daily campaign metrics; weekly attribution models
Sensitive: Ad identifiers and audience membership (no raw PII)
Markets, Partners & Risk
Support market-entry decisions, partner due diligence and risk monitoring
Key metrics: Market Opportunity Score, Partner Pipeline Value, Control Status, Incident Count
Access: CEO, Strategy team, Legal/compliance, Board (summary)
Refresh: Monthly market indicators; event-driven incidents
Sensitive: B2B contacts and due-diligence records (restricted)
Growth Path
Scalability Roadmap — Posture, Infrastructure and Team Shape
Design intent, not committed spend — the A$46.43/mo run cost buys none of the Stage 1–3 infrastructure below.
Stage 1
Serverless, scheduled processing, one BI environment
Infrastructure: Single AWS account; S3, Glue, Redshift Serverless, QuickSight
Team: 2–3 data engineers (can be contractors); fractional architect
Trigger to advance: Prove reconciled metrics and demonstrate user value
Stage 2
Autoscaling ingestion, separated dev/test/prod, stronger observability
Infrastructure: Multi-environment AWS; dedicated networking; monitoring and alerting
Team: 4–6 data/platform engineers; dedicated BI developer; data analyst
Trigger to advance: Add availability guarantees, private networking and formal on-call support
Stage 3
Multi-account, queue buffering, load testing, regional recovery design
Infrastructure: Multi-account AWS; cross-region recovery; CDN; advanced security
Team: 8–12+ platform, data and security engineers; dedicated governance
Trigger to advance: Dedicated platform/security team and negotiated cloud commitments
Procurement
Technology Options by Layer
Three products per layer, priced on the basis shown — the workload that selects between them is settled at gate G2.
| Layer | Recommended | Alternative | Premium Alternative | Pricing Basis |
|---|---|---|---|---|
| Source Contracts | Open JSON schemas | AWS Glue Schema Registry | Confluent Schema Registry | Internal labour / managed usage |
| Batch Ingestion | AWS-native jobs | Airbyte (open-source) | Fivetran (MAR-based) | Compute / infra / MAR |
| Streaming | Amazon Kinesis | Amazon MSK (Kafka) | Confluent Cloud | Throughput / storage |
| Landing/Raw Storage | Amazon S3 | Azure Data Lake Storage Gen2 | Google Cloud Storage | Storage + requests |
| Processing | AWS Glue/Spark | Databricks | Snowflake Processing | Job compute / DBU / credits |
| Warehouse/Query | Redshift Serverless + Athena | Snowflake | BigQuery | RPU / credits / bytes |
| Orchestration | AWS Step Functions + dbt Core | Prefect | dbt Cloud | Usage / workspace / seats |
| Governance | Lake Formation + Glue Catalogue | Databricks Unity Catalog | Alation/Collibra | Cloud use / platform / quote |
| BI | Amazon QuickSight | Power BI | Looker | User / session / capacity |
| AI/API | SageMaker + API Gateway | Databricks ML | BigQuery ML + API | Compute / inference |
Governance
Technology Approval Gates — No Gate, No Spend
Five engineering gates, TG-0 to TG-4 — numbered apart from the financial decision schedule, which runs G0–G2 only.
TG-0: Entity and Rights
Certified entity, ownership, IP, data and contract rights
If failed: Stop all work
TG-1: Data Feasibility
Representative extracts, data dictionaries, control totals and consent samples
If failed: Do not build
TG-2: MVP Value
Three certified dashboards, reconciliation above agreed threshold, named users
If failed: Fix or stop
TG-3: Production
Security review, recovery test, retention policies, vendor DPAs, runbook
If failed: No production PII
TG-4: Scale
Demonstrated unit economics, reliability and procurement case
If failed: Avoid committed licences
Compliance by Design
Retention & Cross-Border Transfers
Retention Policy
| Data Type | Retention | Rationale |
|---|---|---|
| Transaction records | 7 years | Tax and audit obligations |
| Customer consent records | Relationship + 3 years | Regulatory evidence |
| Marketing campaign data | 3 years | Performance analysis and attribution |
| Raw Bronze data | 3 years minimum | Reprocessing capability |
| Operational logs | 2 years | Security and incident investigation |
| Deletion request records | Permanent (metadata only) | Proof of compliance |
Transfer Mechanisms
| Route | Legal Mechanism |
|---|---|
| Australia to EU | Standard Contractual Clauses (SCCs) or adequacy decision |
| Australia to UK | UK International Data Transfer Agreement or UK SCCs |
| Australia to USA | EU-US Data Privacy Framework (participating vendors); SCCs otherwise |
| Australia to Canada | APP 8 (Privacy Act 1988 (Cth)) reasonable-steps assessment before disclosure |
| Any transfer | Transfer Impact Assessment required before production data flows |
Committed MVP Run Cost
A$46.43/mo
CalculatedThe committed MVP bill of materials, monthly-cancellable.
