Why Our Messages Don't Get Lost or Reordered

This is not a marketing post. Every guarantee maps to auditable code, tests, and documentation in the repository. Clone it and verify for yourself — reproduction commands at the end.

“Will messages get lost? Will they arrive out of order?” — these are the first questions any technical decision-maker asks when evaluating an IM solution, and the hardest to get honest answers to. Most vendors reply with four words — “no loss, no reordering” — and leave it at that.

This article dissects IMCore’s answer layer by layer: what each layer guarantees, how it works, and how to verify it.

The one-sentence answer

Messages are persisted to the database before the server acknowledges them; within a conversation, ordering is enforced by a database-level lock; after reconnection, clients catch up precisely via cursor-based backfill.

Now let’s unpack each layer.

Layer 1: Messages don’t get lost — write to disk first, then acknowledge

The most common way messages get lost

Most IM systems handle sends like this:

Client sends → Server receives → Push to recipients → Return "success"

The problem: what if the server crashes after pushing to the recipient but before returning “success”? The client thinks the send failed and retries — but the recipient already got a “ghost message.” Conversely, if the server crashes before pushing, the client gets a timeout — but the message may already be in a memory queue, never to be seen again.

IMCore’s approach: synchronous idempotent insert

Client sends
    ↓
Server generates UUIDv7 message ID
    ↓
Synchronous INSERT into PostgreSQL (idempotent: retries don't duplicate)
    ↓ ←—— Data is on disk. Power loss here doesn't lose the message.
Push to online recipients
    ↓
Return "success" to sender

The critical detail: INSERT happens before ACK. The moment the client receives “success,” the message is already in the database. Crash, restart, process killed — the message survives.

Idempotency comes from the database unique key (conversation_id, sender_id, client_dedup_id) — a client retrying the same message (network retry, reconnection resend) hits INSERT IGNORE, which doesn’t create a second row and returns the original message_id.

-- PostgreSQL
INSERT INTO dm_messages (conversation_id, message_id, sender_id, client_dedup_id, payload, ...)
VALUES (?, ?, ?, ?, ?, ...)
ON CONFLICT (conversation_id, sender_id, client_dedup_id) DO NOTHING;

-- MySQL
INSERT IGNORE INTO dm_messages (conversation_id, message_id, sender_id, client_dedup_id, payload, ...)
VALUES (?, ?, ?, ?, ?, ?);

On resend, RowsAffected = 0 — the server looks up the original message_id and returns inserted=false, so the client knows this was a retry and doesn’t need to re-render it.

Layer 2: No reordering — same-conversation ordering enforced by a database lock

The classic reordering scenario

Alice and Bob send messages to the same group chat almost simultaneously:

Time T1: Alice's INSERT gets id=100, but the transaction hasn't committed yet
Time T2: Bob's   INSERT gets id=101, and commits first
Time T3: A recipient pulls messages with id > 99 → sees 101, can't see 100
Time T4: Recipient advances cursor to 101
Time T5: Alice's transaction commits, id=100 becomes visible
Result: Recipient's next pull is id > 101 → empty. Message 100 is permanently skipped.

This isn’t hypothetical — we found and fixed this exact bug in our own codebase.

IMCore’s fix: the conversation write lock

// lib/dblib/conversation_write_lock.go
func withConversationWriteLock(ctx context.Context, key string, fn func(conn *sql.Conn) error) error {
    // PG: pg_try_advisory_lock(hashtextextended(key, 0))
    // MySQL: GET_LOCK(key, 0)
    //
    // Try-and-backoff loop: waiters hold no database connection
    // Holder runs the INSERT on the same connection; lock releases after COMMIT
    // → Within one conversation, "assign id" and "commit" are strictly serialized
}

Effect: within a single conversation, id assignment order == commit order. In the scenario above, Bob’s INSERT waits for Alice’s COMMIT to finish before getting its id — a recipient can never see 101 without seeing 100 first.

This isn’t a convention — it’s enforced by database-level advisory locks. We wrote a deterministic regression test (lib/dblib/conversation_write_lock_test.go) that pins this behavior: when an external holder owns the lock, the DAO’s insert must block; after release, it must complete.

Why not a centralized sequence number?

A centralized per-conversation sequence (one Redis INCR per conversation) introduces extra latency and infrastructure dependency under high concurrency. Our approach:

• Message ID = UUIDv7 (client-side cursor; lexicographic order ≈ chronological)
• Database auto-increment id = server-side total order (history pagination; naturally unique)
• Conversation write lock = guarantees id assignment order == commit order (no extra sequence number needed)

Zero additional infrastructure — pure database-native capabilities.

Layer 3: Reconnection recovery — precise cursor-based catch-up

Realtime pushes can drop (client disconnects, network jitter), but as long as the message is in the database, the client can always catch up.

The mechanism

Client's last-seen message ID: "msg-019abc..."
    ↓
After reconnecting, sends historyPage({ afterID: "msg-019abc..." })
    ↓
Server: maps UUIDv7 → database id → SELECT ... WHERE id > ? ORDER BY id ASC
    ↓
Returns every missing message (precise to the row, no more, no less)

Key design decision: afterID is a message the client has actually seen — not the server guessing “where should we be.” The client only trusts what it has witnessed; the server only sends what came after — no ambiguity.

SDK-level coordination

All 10+ platform SDKs automatically handle the reconnected event:

1. Flush the offline queue (only after login confirmation)
2. Incremental backfill per conversation via cursor
3. Enforce lastMessageSeq monotonic guard (stale events can’t overwrite newer snapshots)

Layer 4: Verification — not “trust us,” but “run it yourself”

Zero-loss verdict

# Under sustained load, kill one instance → survivors take over → restart the killed one → compare sent == persisted
scripts/chaos/graceful_drain.sh
scripts/chaos/verify_no_loss.sh --report <loadtest-report> --room <room-id>

Ground truth is database row count, not delivery counters — sent == COUNT(*) settles it in one comparison.

Session recovery matrix

Nine automated scenarios (SR-01 through SR-09) cover: initial fetch racing with realtime pushes, reconnection backfill, multi-device concurrent read marking, conversation cleanup after group removal, concurrent senders with interleaved commits (our most recent addition) — each running continuously in CI.

Load test reproduction

# 4C8G cloud ECS, 10,000 concurrent connections, 200 msg/s, 50 rooms
# All steps: 0 errors, 0 drops, 0 slow-consumer closes, p99 < 200ms
# The saturation tool automatically provisions and destroys ECS instances
ALICLOUD_REGION=cn-hangzhou LT_SAT_STEPS=5000,6000,8000,10000 \
  bash scripts/loadtest_saturation.sh

Honest boundaries

The following scenarios are not covered by these guarantees:

The debugging story: how we found the reordering bug

In October 2026, while comparing cloud load-test data between old and new code, we noticed the 10,000-connection p99 had climbed from 139ms to 262ms. Using automated bisection across 561 commits, we pinpointed the culprit: a “performance optimization” that replaced the per-member conversation-state flow with whole-room batched pipelines — a huge win in local benchmarks, but in production topology the mega-pipelines head-of-line blocked the Redis event loop.

The lesson (for us, and for you): performance regressions aren’t scary — what’s scary is not having the instruments to detect them. We spent a full day building the complete diagnostic chain from A/B comparison to automated bisection — that chain is now part of the product.

Summary

IMCore ships as binaries by default, with source access available under license. If you have questions about any layer described here, talk to us — we’ll answer with the exact code path and the test that pins it, not marketing copy.