One reliable crypto price feed from many unreliable sources
Crypto market data is fragmented. Every provider has its own latency, rate limits and pricing, and none of the affordable ones gave this fintech startup (under NDA) what it needed: dashboards that update about every 15 seconds. The public APIs within reach updated every 30–40 seconds, and not consistently.
We were part of a three-person full-stack team that took the crypto market data aggregator from MVP to production over five months.
Key features
- Staggered multi-provider polling: each provider is polled on its own offset schedule, so requests are spread out instead of hitting rate limits all at once.
- Tick deduplication and reconciliation: duplicate and conflicting price ticks are merged into a single source of truth.
- Weighted provider selection: providers are weighted, so the aggregator prefers more dependable sources when choosing which data to trust.
- Automatic fallback feeds: when one source degrades, data keeps flowing from the others.
- ~15-second consolidated updates: a Redis-cached feed refreshes on a cycle of roughly 15 seconds.
- WebSocket delivery: dashboards receive new prices as they are published, without polling the server.
- Cost-aware scheduling: polling schedules are tuned to keep paid API calls to a minimum.
Architecture and tech stack
Aggregation layer on Laravel and PostgreSQL
The aggregator is a Laravel application backed by PostgreSQL. Scheduled polling jobs collect prices from each provider and pass the ticks to reconciliation. Providers answer at different speeds, so the reconciliation logic is built to handle late data rather than assume every source reports at the same moment.
Redis cache and WebSocket feed
Reconciled prices are written to Redis, which serves as the consolidated feed. Dashboards read from that cache and never call providers directly, so more dashboard traffic doesn't mean more paid API calls. New prices reach the frontend over WebSockets, which keeps the dashboards live.
Inertia.js pages and Docker
User-facing pages are built with Inertia.js, with server-side rendering on the pages where search visibility mattered. The whole system is containerized with Docker, so deployments are repeatable.
Reliability and cost control
The hard problems here were operational:
- Latency variance: reconciliation absorbs the gap between fast and slow providers, so one slow source doesn't hold back the whole feed.
- Rate limits and data costs: staggered polling and caching keep request volume within each provider's limits and keep external API spend down.
- Provider outages: with weighted selection and fallback feeds, losing one provider leaves fewer sources behind the feed instead of taking the product down.
Related work
The AI Crypto Intelligence Platform adds sentiment analysis and scam detection on top of market data. Finkonkurs is a crypto prices, markets and news portal. We build data-heavy products like these as part of our web app development service.
