Skip to content

fix: handle Gmail raw response streams correctly - #3271

Draft
DenBond7 wants to merge 1 commit into
masterfrom
issue_3270_fix_gmail_raw_attachment_and_MIME_stream
Draft

fix: handle Gmail raw response streams correctly#3271
DenBond7 wants to merge 1 commit into
masterfrom
issue_3270_fix_gmail_raw_attachment_and_MIME_stream

Conversation

@DenBond7

Copy link
Copy Markdown
Collaborator

Description

Gmail returns raw MIME messages and attachment data as Base64URL values inside JSON responses. We use custom FilterInputStream implementations to skip the JSON prefix and stop reading at the closing quote.

The current implementations do not correctly handle every valid InputStream read pattern.

Problems

  • read() is not overridden, so byte-by-byte consumers can receive the closing quote and trailing JSON.
  • read(byte[], off, len) searches the entire buffer instead of only the newly read range. Existing buffer content containing " can therefore cause premature end-of-stream.
  • With a non-zero offset, the returned byte count is based on the quote’s absolute buffer index instead of the number of bytes read from off.
  • Encountering the closing quote at the start of a read can return 0 for a non-empty request, violating the InputStream contract.
  • The JSON prefix is skipped with a single skip() call, although InputStream.skip() may consume fewer bytes than requested.
  • Reads after the closing quote are not explicitly kept at end-of-stream.

These issues can cause MIME messages or attachments to be truncated, incorrectly decoded, or contaminated with trailing JSON. The behavior depends on the delegate stream, buffer contents, offsets, and read chunk size.

Steps to reproduce

Byte-by-byte read

  1. Create a Gmail raw MIME response such as:
{
  "raw": "YWJjZA=="
}
  1. Wrap it in GMailRawMIMEMessageFilterInputStream.
  2. Read the stream using read() one byte at a time.
  3. The closing quote and JSON suffix may be returned after the Base64URL value.

Reused buffer with offset

  1. Fill a byte buffer with values that include a quote.
  2. Call read(buffer, nonZeroOffset, length).
  3. Observe that content outside the newly read range can affect delimiter detection and the returned byte count.

Partial prefix skip

  1. Use a delegate stream whose skip() method consumes only one byte per call.
  2. Construct GMailRawAttachmentFilterInputStream.
  3. Observe that the complete JSON prefix is not skipped.

Expected behavior

  • Only the Base64URL value is exposed.
  • Both read() and read(byte[], off, len) behave consistently.
  • The closing quote terminates the filtered stream.
  • Every read after the closing quote returns -1.
  • Buffer offsets and lengths follow the InputStream contract.
  • The entire JSON prefix is consumed even when the delegate performs partial skips.
  • A truncated prefix produces a clear EOFException.

close #3270


Tests (delete all except exactly one):

  • Does not need tests (refactor only, docs or internal changes)
  • Difficult to test (explain why)
  • Not worth testing
  • Tests will be added later (issue #...)
  • Tests added or updated

To be filled by reviewers

I have reviewed that this PR... (tick whichever items you personally focused on during this review):

  • addresses the issue it closes (if any)
  • code is readable and understandable
  • is accompanied with tests, or tests are not needed
  • is free of vulnerabilities
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

1 participant