ExWapp.Proto.HistorySync (ExWapp v0.1.2)

Copy Markdown View Source

Extracts WhatsApp history sync payloads embedded in protocol messages.

Mirrors the flow used in reference clients:

  • Message.protocolMessage.historySyncNotification
  • initialHistBootstrapInlinePayload (zlib-compressed)
  • HistorySync.conversations

Decoding uses the generated WAWebProtobufsHistorySync modules; this module only flattens the structs into the internal map contract.

Inline chunks versus blob chunks

WhatsApp delivers the bootstrap chunk inline (initialHistBootstrapInlinePayload), and the larger RECENT/FULL chunks as encrypted media the client is expected to download. Only the inline path is implemented: the bootstrap gives roughly the latest message per chat, which is enough for a chat list but not for reading conversation history.

Blob chunks are not silently discarded — extract_from_message/1 returns {:pending_blob, blob_ref()} with everything a download needs, so the gap is a missing consumer rather than lost data. To close it:

  1. teach ExWapp.Media.Crypto the :history type — HKDF context string "WhatsApp History Keys" (whatsmeow download.go), and allow it in validate_ref/1;
  2. hand the blob_ref() to the existing ExWapp.Media.HTTP.download/3, which already resolves media_conn from direct_path and verifies the HMAC;
  3. zlib-inflate the plaintext and feed it to parse_history_payload/2 below — the parsing and flattening already work, they are what the inline path uses.

Run the download off the inbound message path (a chunk can be megabytes); whatsmeow uses a dedicated worker with an idle timeout for exactly this reason.

Summary

Types

blob_ref()

Everything needed to download and decrypt a history-sync chunk delivered as media instead of inline. Kept verbatim from the notification so a future downloader needs no re-parse.

conversation()
conversation_message()
t()

Functions

extract_from_message(message_payload)
parse_history_payload(payload, sync_type_from_notification)

Flattens an already-inflated HistorySync payload into the internal contract.

Types

blob_ref()

@type blob_ref() :: %{
  sync_type: atom() | non_neg_integer() | nil,
  chunk_order: non_neg_integer() | nil,
  progress: non_neg_integer() | nil,
  oldest_msg_timestamp: non_neg_integer() | nil,
  media_key: binary() | nil,
  direct_path: String.t() | nil,
  enc_handle: String.t() | nil,
  file_sha256: binary() | nil,
  file_enc_sha256: binary() | nil,
  file_length: non_neg_integer() | nil
}

Everything needed to download and decrypt a history-sync chunk delivered as media instead of inline. Kept verbatim from the notification so a future downloader needs no re-parse.

conversation()

@type conversation() :: %{
  jid: String.t(),
  name: String.t() | nil,
  unread_count: non_neg_integer() | nil,
  last_message_timestamp: non_neg_integer() | nil,
  pinned: boolean() | nil,
  archived: boolean() | nil,
  muted_until: non_neg_integer() | nil,
  is_group: boolean(),
  pn_jid: String.t() | nil,
  lid_jid: String.t() | nil,
  tc_token: binary() | nil,
  tc_token_timestamp: non_neg_integer() | nil,
  tc_token_sender_timestamp: non_neg_integer() | nil,
  messages: [conversation_message()]
}

conversation_message()

@type conversation_message() :: %{
  id: String.t() | nil,
  from_me: boolean(),
  timestamp: non_neg_integer() | nil,
  text: String.t() | nil,
  participant: String.t() | nil,
  remote_jid: String.t() | nil,
  order_id: non_neg_integer() | nil
}

t()

@type t() :: %{
  sync_type: atom() | non_neg_integer() | nil,
  conversations: [conversation()],
  lid_mappings: [{String.t(), String.t()}],
  call_log_records: [ExWapp.Call.t()]
}

Functions

extract_from_message(message_payload)

@spec extract_from_message(binary()) ::
  {:ok, t()}
  | {:pending_blob, blob_ref()}
  | :not_history_sync
  | {:error, term()}

parse_history_payload(payload, sync_type_from_notification)

@spec parse_history_payload({:ok, binary()} | binary(), term()) ::
  {:ok, t()} | {:error, term()}

Flattens an already-inflated HistorySync payload into the internal contract.

The inline path calls this after inflating the bootstrap payload; a future blob downloader can call it with the decrypted-and-inflated chunk instead.