Design a Payment System
On this page
- The Domain Invariant: Authorization ≠ Settlement
- Actors and System Boundary
- External entities (outside our system)
- In scope (our system)
- Requirements
- Functional
- Non-functional
- Out of scope (say this in interviews)
- Assumptions and Back-of-the-Envelope Math
- High-Level Design
- API Surface
- Wallet vs Ledger
- Reconciliation
- Fraud and Risk (Decoupled)
- Reliability Under Retries and Timeouts
- How the Design Meets Requirements
- What Breaks When…
- Common Mistakes I Made
- Key Takeaways
- Test Your Understanding
Design a Payment System
A merchant called us: checkout showed "payment approved," the customer got a confirmation email, but the money never landed in their account. Support pulled the transaction — authorized: yes, captured: no, settled: pending. The customer's card had a hold; our ledger had a gap; the payment service provider (PSP) daily settlement file did not match our internal totals.
I knew idempotency and async processing in the abstract. I had not internalized the invariant that makes payments bite: authorization and settlement are different phases. Approval reserves funds; settlement moves money. Skip that split in the design and support ends up re-explaining card holds while finance reconciles against a PSP file that does not match your ledger.
Below is how I whiteboard a card platform: actors, requirements, napkin math, authorize vs capture APIs, wallet vs ledger, daily reconciliation, and what happens when the PSP times out while the client retries.
Intended audience: developers past the fundamentals who want one composed fintech case — interview whiteboard or an internal design review, not a glossary dump.
Prerequisites:
- Reliability and Fault Tolerance — retries, timeouts
- Asynchronous Processing and Messaging — queues, at-least-once
- Idempotency and Delivery Guarantees — idempotency keys, outbox
- Load Balancing and Cross-Cutting Concerns — high availability, PCI, observability
- Helpful: Event Sourcing and CQRS — append-only ledger mental model
The Domain Invariant: Authorization ≠ Settlement
Card payments are a two-phase process:
| Phase | What happens | Speed | Money moves? |
|---|---|---|---|
| Authorization | Issuer validates card, checks limit/fraud, returns auth code | Seconds | No — amount is reserved |
| Capture / settlement | Capture confirms the charge; settlement moves funds to the merchant (often batched daily) | Hours to days | Yes |
That split drives everything else:
- You need authorize and capture as separate API steps (or explicit state machine)
- A txn can be authorized but not captured — fulfillment vs payout decisions
- Idempotency matters on both steps — timeouts during authorize are ambiguous (idempotency)
- Reconciliation compares your ledger to the PSP's settlement file — auth success does not guarantee settlement success
If you remember one thing for interviews: approval ≠ paid.
Actors and System Boundary
External entities (outside our system)
| Entity | Role |
|---|---|
| Customer | Cardholder purchasing goods/services |
| Merchant | Seller; online store collects payment details |
| Issuer bank | Customer's bank; validates card and credit limit |
| Acquiring bank | Merchant's bank |
| Payment service provider (PSP) | Third-party gateway (Stripe, Adyen, etc.) — routes to card networks; carries most PCI scope |
| Card network | Visa, Mastercard — network rules and validation |
We do not build banks or card networks. We orchestrate calls to the PSP and keep our own wallet, ledger, and reconciliation state.
In scope (our system)
| Component | Role |
|---|---|
| Payment service | Orchestrates payment flow, persists events |
| Fraud / risk check | Real-time scoring before forwarding to PSP |
| Wallet | Mutable merchant balance (operational) |
| Ledger | Immutable append-only audit trail |
| Reconciliation | Match ledger vs PSP settlement files |
| Dispute service | Chargebacks, compliance records |
Loading diagram…
Requirements
Functional
| Requirement | Notes |
|---|---|
| User registration and authentication | Merchant-facing; hash passwords, session management |
| Payment processing | Credit/debit via PSP; authorize then capture |
| Transaction history | By customer and date range |
| Balance management | Merchant available + pending balances |
| Mobile accessibility | Responsive merchant UI |
Non-functional
| Requirement | Target |
|---|---|
| Security | Payment Card Industry (PCI DSS); encrypt card data in transit; never log primary account number (PAN) or card verification value (CVV) |
| Availability | Highly available payment path; no single point of failure on orchestration |
| Reliability | Redundant services, multi-datacenter deployment, failover |
| Scalability | Handle peak sale events; horizontal scale orchestration tier |
| Performance | Low-latency authorize path; async settlement acceptable |
Out of scope (say this in interviews)
- Building our own card network or becoming a licensed bank
- Cryptocurrency rails
- Full accounting / tax reporting (export to finance systems only)
- In-person point-of-sale (POS) hardware
Assumptions and Back-of-the-Envelope Math
Assumptions (state these before calculating):
- 50M transactions per day
- ~100 bytes per transaction record; ~200 bytes per user record (stored once)
- Peak ≈ 2.5× average → ~1,400 TPS at peak
- ~130 bytes average response (status, tokens, metadata)
Storage (per day):
50M × 100 B ≈ 5 GB/day in transaction events (plus indexes, ledger entries)
User records are amortized — transactions reference user_id.
Bandwidth at peak (~1,400 TPS):
Incoming: 1400 × 100 B ≈ 1.1 Mbps
Outgoing: 1400 × 130 B ≈ 1.5 Mbps
Real deployments need headroom for retries, webhooks, reconciliation files, and metadata — multiply by 2–5× for planning.
Servers (illustrative):
If one app server handles ~64K RPS for lightweight health checks but payment orchestration is heavier, do not use health-check RPS for payment pods. Payment path is I/O-bound (PSP latency dominates). Start with horizontal scale behind a load balancer; size from p99 PSP round-trip, not CPU alone.
When math changes the design:
- 5 GB/day → time-partitioned txn tables or cold archive; ledger never deleted
- Peak TPS → async capture pipeline acceptable; authorize path stays sync
- Global merchants → multi-region reads; writes sticky or single primary (multi-region)
High-Level Design
- Customer clicks Pay on merchant site → payment service receives event
- Payment service → fraud/risk check (decoupled service)
- On pass → PSP → issuer bank authorization
- Auth code returned → merchant can fulfill (state:
AUTHORIZED) - Later: capture job or API → PSP captures funds → wallet updated
- Ledger append for every state transition
- End of day: reconciliation vs PSP settlement file
Fraud sits in its own service so weekly rule changes are not tied to PSP release windows, and a bad fraud deploy does not take down capture.
API Surface
Interviewers expect a two-step payment API because of authorization vs capture:
| API | Purpose |
|---|---|
registerUser(username, email, password) | Hash password, store user |
authenticateUser(username, password) | Session / token |
authorizePayment(amount, card, merchant_id, idempotency_key) | Reserve funds; return authorization_code |
capturePayment(authorization_code, amount, idempotency_key) | Move funds to merchant |
checkPaymentStatus(transaction_id) | Poll for async states |
getTransactionHistory(customer_id, start, end) | History |
Pass idempotency_key on authorize and capture. The client generates one
key per checkout intent; the server stores key → result and returns the same
response on retry. Storage patterns and TTL: Idempotency and Delivery Guarantees.
// Client: one key per checkout intent
const idempotencyKey = `checkout-${cartId}-${userId}`;
await fetch('/payments/authorize', {
method: 'POST',
headers: { 'Idempotency-Key': idempotencyKey },
body: JSON.stringify({ amount, cardToken, merchantId }),
});
Wallet vs Ledger
| Wallet | Ledger | |
|---|---|---|
| Purpose | Operational balance — "how much can merchant withdraw?" | Audit trail — "what happened, when, between whom?" |
| Mutability | Updated in place on capture/refund | Append-only; no updates or deletes |
| Source of truth for | Payout decisions | Disputes, finance, reconciliation |
| Analogy | Checking account balance | Bank statement line items |
Flow: capture succeeds → update wallet → append ledger entry (txn_id, amount, parties, status, timestamp).
If wallet and ledger disagree, ledger wins for audit; wallet is rebuilt from ledger in recovery. Event sourcing thinking helps here — even if you do not run full event sourcing, treat the ledger as the immutable log.
Reconciliation
Every day the PSP sends a settlement file summarizing captured transactions and fees.
Reconciliation job:
- Load internal ledger entries for
status = CAPTUREDfor the settlement window - Compare totals and line items to PSP file
- Flag mismatches (missing capture, amount drift, duplicate settlement)
Example mismatch: merchant ledger shows three payments totaling $225; PSP file shows $200 — investigate before payout.
Reconciliation is not optional for production fintech. Auth success without matching settlement is the incident that started this post.
Fraud and Risk (Decoupled)
Fraud detection — pattern analysis (velocity, device, geo)
Risk check — score → proceed / challenge (3-D Secure, 3DS) / block
Why separate from PSP?
- Different release cadence (rules change weekly)
- Different blast radius (bad deploy shouldn't take down PSP path)
- Scale independently — Black Friday traffic spikes fraud checks before PSP limits
Fallback trade-off: if fraud service times out on a $2 charge, some teams fail open (accept risk); on $2,000, fail closed. Document the policy — interviewers ask.
Reliability Under Retries and Timeouts
I point to earlier posts for the mechanics here instead of re-teaching Kafka in the interview.
| Failure | Pattern | Where to read more |
|---|---|---|
| Lost payment events | Pub-sub (Kafka); ack after wallet + ledger write | Async processing |
| Transient PSP errors | Retry + exponential backoff + max attempts | Reliability |
| Client timeout on authorize | Idempotency key — ambiguous if charge went through | Idempotency |
| Double-click Pay | Same idempotency key on authorize | Idempotency |
| Fraud service down | Explicit fail-open vs fail-closed policy | Fraud section above |
Loading diagram…
How the Design Meets Requirements
Before I stop drawing boxes, I map each requirement row to something buildable — not a label on a diagram.
| Requirement | Technique |
|---|---|
| User auth | Merchant app + payment service validates session |
| Payment processing | Payment service + PSP + fraud; authorize → capture |
| Transaction history | Ledger query by customer_id + date range |
| Balance | Wallet updated on capture; pending = authorized-not-captured |
| Mobile | Responsive merchant UI |
| Security | PCI scope via PSP tokenization; no PAN in our DB |
| Availability | Redundant payment/recon services; multi-datacenter; load balancing |
| Scalability | Async capture; horizontal payment pods; DB partitioning by merchant_id |
| Performance | Sync authorize (user-facing); batch settlement (background) |
What Breaks When…
-
Authorization succeeds, settlement fails — customer sees hold; merchant not paid. Mitigation: txn status machine, reconciliation alerts, automated capture retry with idempotent capture keys.
-
Timeout after authorize — client retries; without idempotency, double auth hold or duplicate capture. Mitigation: idempotency store with 24h TTL minimum.
-
Ledger–PSP mismatch at reconciliation — payout blocked until ops resolves. Mitigation: daily recon job, alerting on any delta > $0, runbook for partial PSP outages.
Common Mistakes I Made
- Treating authorize and capture as one "charge" API — breaks retries and state tracking
- Updating wallet without ledger append — can't explain balance to finance after a bug
- Skipping reconciliation until finance finds a hole — build day-one batch job
- Logging card numbers during debugging — PCI scope violation; use PSP tokens only
- Re-explaining Kafka in the interview instead of naming one sentence and moving to wallet/ledger
Key Takeaways
- Authorization ≠ settlement — design APIs and state machines around two phases
- Wallet is operational; ledger is audit — append-only ledger for every money movement
- Idempotency keys on authorize and capture — timeouts are normal; duplicate charges are not
- Reconciliation is production-critical — compare ledger to PSP settlement daily
- Decouple fraud from PSP — scale and deploy independently; document fail-open policy
- Compose fundamentals, don't re-lecture — load balancing, async, idempotency, and observability show up here by reference
Test Your Understanding
Next in series: Fan-Out on Write vs Fan-Out on Read
Happy coding!