0

How I Built A Real-time Streaming Market Data with .NET and ReactJS

Architecture of RealMarketAPI

Last month I shared how I turned a side project into a SaaS. Today, I’m opening the hood and showing you the actual architecture, tech decisions, and lessons learned.


📋 Table of Contents


I. Why I Built This

I got tired of:

  • Paying $200–$800/month for APIs that go down during high volatility
  • Inconsistent latency (especially WebSocket)
  • Terrible historical data quality
  • Vendors suddenly changing pricing or throttling indie developers

So I decided to build my own — focused on reliability, transparency, and developer experience.


II. High-Level Architecture

Here’s the current system:

Architecture of RealMarketAPI


III. Core Components Explained

1. RealMarketAPI (Entry Point)

  • Simple REST + WebSocket gateway
  • Handles authentication (JWT + API keys)
  • Rate limiting & usage tracking

2. Auth Service

  • Validates tokens
  • Checks subscription tier
  • Rejects invalid requests early

3. RealMarketServices (The Brain)

This is where the magic happens. It’s split into three main services:

  • Ticker Service – Ultra-low latency price updates
  • Stream Service – WebSocket broadcaster (using Redis Pub/Sub)
  • Historical Service – Candle & tick data delivery

4. Data Storage Strategy

  • Redis → Hot short-term ticker data + Pub/Sub
  • PostgreSQL + TimescaleDB → Long-term candles and relational data
  • Candles DB → Dedicated time-series optimized storage

5. Data Ingestion

  • Multiple workers fetch from various Brokers/Exchanges
  • Retry logic + circuit breakers + fallback sources
  • Data is normalized into a unified format

6. RealMarketBots (Bonus Layer)

One of my favorite parts:

  • Scrapes news & trends using RSS + SerpAPI
  • Generates content summaries with Gemini
  • Sends Telegram alerts
  • Auto-publishes to Facebook and X

NextJS

IV. Tech Stack

Backend: C# .NET

using FastEndpoints;
using RealTimeMarketAPI.Core.Modules.TickerModule.Handlers;
using RealTimeMarketAPI.Infrastructure.Models;

namespace RealTimeMarketAPI.Core.Modules.TickerModule.Endpoints
{
    public class GetCandlesEndpoint : Endpoint<GetCandlesRequest, ListResult<GetCandlesResponse>>
    {
        public override void Configure()
        {
            AllowAnonymous();
            Get("candle");
            Version(1);
        }

        public override async Task HandleAsync(GetCandlesRequest req, CancellationToken ct)
        {
            var result = await req.ExecuteAsync(ct);
            await Send.OkAsync(result, cancellation: ct);
        }
    }
}

Stack using: FastEndpoints, PostgreSQL, Redis

Frontend: Next.js
NextJS

  • Real-time: Redis + custom WebSocket server
  • Database: PostgreSQL
  • Observability: Datadog
    Datadog for monitoring

V. Biggest Challenges So Far

  • WebSocket Scale – Maintaining thousands of concurrent connections with low latency is hard.
  • Data Consistency – Different exchanges have different timestamp formats and precision.
  • Cost Control – Exchange API fees + bandwidth can explode quickly.
  • Testing – You can’t easily replay real market conditions.

VI. What’s Next

  • TradingView widget
  • Official SDKs (Python, Node.js, Go)
  • More symbols & exchanges
  • Dedicated enterprise nodes

VII. Final Words

Building a market data api taught me one important lesson: Reliability beats features.

Most users don’t need 10,000 symbols. They need 3 symbols that actually work at 3 AM when Bitcoin crashes.

That’s what I’m optimizing for.

You can check it out here: RealMarketAPI


Let me know in the comments:

  • What market data pain point do you have?
  • Would you use this for algo trading, dashboards, or AI agents?

I read every comment


All rights reserved

Viblo
Hãy đăng ký một tài khoản Viblo để nhận được nhiều bài viết thú vị hơn.
Đăng kí