Point LiteJoin at your APIs, join data from multiple sources with SQL, and get live results instantly. No Kafka required. No Flink. No cluster to manage.
$ curl -sSL https://litejoin.dev/install | sh
No event bus to set up. No streaming infrastructure to learn. Just URLs and SQL.
Point LiteJoin at any REST API, webhook, or event stream. It polls, detects changes, and emits only the deltas — turning any request/response API into a live stream automatically.
→Write standard SQL to join streams together. Enrich orders with user profiles. Combine IoT readings with device metadata. LiteJoin executes joins reactively — results emit the instant new data arrives.
→Consume joined results via SSE, HTTP webhooks, or Kafka. Power live dashboards, trigger workflows, feed downstream services — all from a single binary running on your terms.
# Turn any REST API into a real-time stream sources: - name: weather type: api url: https://api.weather.gov/stations/KNYC/observations/latest interval: 30s topic: weather key_path: $.id change_detection: diff - name: github_prs type: api url: https://api.github.com/repos/org/repo/pulls interval: 60s topic: pull_requests key_path: $.number headers: Authorization: Bearer ${GITHUB_TOKEN}
Most APIs don't push changes to you. LiteJoin polls them, diffs against previous state, and emits only what changed. No webhook setup. No event bus. Just a URL.
Sources, joins, windows, and sinks — all in a single YAML config. LiteJoin reads it and runs the pipeline. No boilerplate services, no orchestration layer, no build step.
sources: - name: orders type: http topic: orders - name: users type: http topic: users joins: - name: order-enrichment sql: | SELECT o.*, u.payload AS user_info FROM orders o LEFT JOIN users u ON json_extract(o.payload, '$.user_id') = json_extract(u.payload, '$.id') triggers: [orders] result_key: $.user_id sinks: - name: live-dashboard type: sse addr: :8081
import { LiteJoin } from 'litejoin'; const lj = new LiteJoin('http://localhost:8081'); // Subscribe to joined results in 3 lines lj.subscribe('order-enrichment', (result) => { console.log(result.user_info.name, result.total); renderDashboard(result); }); // Or use the REST API for materialized views const latest = await lj.query('order-enrichment', { where: { region: 'us-east' } });
The LiteJoin JS SDK connects to your SSE sink and delivers typed, parsed results directly to your frontend. No WebSocket boilerplate. No polling. Just a callback.
Config to live dashboard. One binary. No moving parts in between.
LiteJoin makes trade-offs that favor simplicity and latency over heavyweight guarantees. Here's exactly what you're getting.
By default, LiteJoin runs in best-effort mode for maximum speed — if a sink is down, results are dropped. Flip one config flag to enable at-least-once delivery: failed results are persisted to a SQLite-backed dead-letter queue and retried with exponential backoff until they land.
Each message is stored in one of 8 shards (configurable), selected by hashing the message key. SQLite WAL mode gives high read concurrency without blocking writes. Data persists to disk — restart without losing state.
Tumbling windows emit on fixed, non-overlapping intervals. Sliding windows overlap and advance by a configurable slide. Session windows close dynamically after an inactivity gap. All support a grace period for late arrivals.
If a polled API goes down, LiteJoin retries on the next poll interval. Joins keep executing against whatever data is in storage. Stale records are evicted by a configurable TTL (default 24 hours) so you don't silently serve old data forever.
Joins fire when new data arrives on a trigger topic. Each batch is processed by a pool of 16 concurrent workers (configurable). Queries run against the SQLite reader pool in parallel. Results emit within milliseconds of ingestion.
A single LiteJoin instance sustains 8,000+ join evaluations per second with 16 workers and 8 SQLite shards on commodity hardware. Throughput scales with shard count and worker concurrency. The bottleneck is almost always your source API, not LiteJoin.
LiteJoin is purpose-built for app developers who need real-time joins without the operational overhead of stream processing infrastructure.
| LiteJoin | Kafka Streams | Apache Flink | Materialize | |
|---|---|---|---|---|
| Setup time | 5 minutes | Hours | Days | ~1 hour |
| Infrastructure required | None — single binary | Kafka cluster | Flink + Kafka cluster | Postgres-compatible server |
| Configuration | One YAML file | Java/Kotlin code | Java/SQL + deployment config | SQL DDL |
| REST API → stream | Built in — just add a URL | Requires Kafka Connect | Requires custom source | Requires webhook bridge |
| Browser streaming | SSE sink + JS SDK | Build your own | Build your own | SUBSCRIBE (Postgres wire) |
| Operational complexity | Run the binary | JVM tuning, partition mgmt | Cluster ops, checkpointing | Managed or self-hosted |
| Best for | App developers | Java backend teams | Data engineering teams | Analytics / BI |
You shouldn't need a data engineering team to get real-time features in your app.
Join your orders API with a delivery fleet API and a weather service. Push enriched, real-time order status to your mobile app over SSE. No webhooks to configure on any of the source systems.
Stream Stripe payment events joined with your user database and product catalog. Compute rolling revenue windows per plan tier, per region, updated every 5 seconds. Display in-app.
Run LiteJoin on an edge device. Join temperature sensor readings with humidity and equipment metadata. Trigger alerts when correlated thresholds are breached. Works offline — single binary, no cloud required.
Poll GitHub, Linear, Slack, and your deploy pipeline API. Join events by engineer and team. Power a single activity feed for your engineering org — built in 15 minutes, not weeks.
LiteJoin is the open-source engine. We're building tools around it.
A visual interface for building streaming pipelines. Set up API sources like Postman requests. Draw joins between them. See results in real time. Export to production YAML when ready.
Managed infrastructure with durability guarantees the open-source engine doesn't provide. Your pipelines, always on.
Install the binary. Write a config. Get real-time joined data flowing to your app.
Watch the 2-minute demo$ curl -sSL https://litejoin.dev/install | sh