bug: `BytecodeMode.FULL` produces duplicate classes + empty DEX files, and `classMap` removal becomes unreliable when combined with other patches

[sjshb57]

Bug description

[Bug] BytecodeMode.FULL produces duplicate classes + empty DEX files, and classMap removal becomes unreliable when combined with other patches

Component

morphe-patcher — BytecodeMode.FULL (BytecodePatchContext.compileFull) and DexReadWrite.writeMultiDexFile

---

Problem 1 — FULL mode emits duplicate classes and empty DEX files

When compiling a large multi-dex app with BytecodeMode.FULL, the output contains extra DEX files filled with duplicate classes:

• The last few output DEX files contain classes that are already present in other DEX files. The duplicated entries are almost exclusively nested inner classes (e.g. Foo$a$b).
• After running a "remove duplicate classes" pass (e.g. in MT Manager), those DEX files shrink to ~140-byte empty files that can be deleted, after which the app runs perfectly.
• In other words, the entire content of these trailing DEX files is a duplicate of classes that already exist elsewhere.

Impact: the user must manually clean up duplicate classes before the APK installs/runs correctly. This step should not be necessary.

Reproduction

1. Compile with BytecodeMode.FULL.
2. Use a large multi-dex app whose class count far exceeds MIN_CLASSES_PER_SEGMENT × MAX_THREADS (so multi-threaded segmentation kicks in).
3. Remove a number of classes via classMap.
4. Build.
5. Inspect the output: the trailing DEX files contain duplicate nested classes; after a duplicate-class cleanup they become empty 140-byte files.

Suspected code (DexReadWrite.kt)

• processSegment(...): after the per-segment loop, writeDexPoolToTemp(currentDexPool, ...) is called unconditionally, so an empty pool still produces an empty DEX file.
• Multi-threaded segmentation (splitIntoMutableSegments + parallel processSegment): on cross-segment class-reference boundaries, some classes appear to be written by more than one segment's DexPool. (Root cause unconfirmed — needs investigation; each class is assigned to a single segment in the split logic, so the duplication is unexpected.)

---

Problem 2 — FULL-mode class removal is unreliable when multiple patches run together

• When a single patch removes classes from classMap under FULL mode, the removal works: the classes are absent from the final output.
• When the same removal runs alongside other patches (multiple bytecode patches in the same build), the removed classes reappear in the final output — they are not deleted.

This suggests an interaction problem in FULL compilation when multiple patches share the same BytecodePatchContext. (Patches execute in name alphabetical order via context.executablePatches.sortedBy { it.name }; the removing patch runs last, yet its classMap.remove does not take effect on the final compiled output in the multi-patch case.)

Reproduction

1. BytecodeMode.FULL.
2. Build with only the class-removing patch enabled → removed classes are gone (correct).
3. Build with the same patch plus other patches enabled → removed classes remain in the output.

---

Expected behavior

• writeMultiDexFile should not emit DEX files containing duplicate classes.
• No empty DEX files should be produced (or empty pools should be skipped).
• classMap removals under FULL mode should reliably take effect, including when multiple patches run in the same build.

Notes

Both problems are specific to BytecodeMode.FULL. STRIP_SAFE/STRIP_FAST do not exhibit them (they only rewrite the small set of modified classes, so segmentation stays single-segment), but FULL is currently the only mode that can fully delete classes from the output, which is why these issues surface there.

Version of Morphe (Manager) and the app you are patching

Environment

• Morphe Manager: 1.21.0-dev.2
• Device: Android 16 (API 36), arm64-v8a
• Target app: large multi-dex application (com.twitter.android, ~203,000 classes, 18 original DEX files)

Morphe logs

- 
This report was actually generated by AI, but I’ve reviewed it thoroughly. The issue was identified in the project I wrote: https://github.com/sjshb57/pairip-patches
If there’s anything else you need me to do, please let me know. I’ll respond promptly. I apologize for the inconvenience.

Acknowledgements

• I have checked all open and closed bug reports and this is not a duplicate.
• I have chosen an appropriate title.
• All requested information has been provided properly.
• The bug only affects using Morphe (Manager)

[sjshb57]

This report was actually generated by AI, but I’ve reviewed it thoroughly. The issue was identified in the project I wrote: https://github.com/sjshb57/pairip-patches
If there’s anything else you need me to do, please let me know. I’ll respond promptly. I apologize for the inconvenience.

[wchill]

Few things:

a) please share a copy of the APK right after patching (use wormhole.app).

b) please try patching with CLI and enable the verification mode, then share the logs from the patching session.

[sjshb57]

Few things:

a) please share a copy of the APK right after patching (use wormhole.app).

b) please try patching with CLI and enable the verification mode, then share the logs from the patching session.

test cli：v1.9.1-dev.3 version

1. Use the following command
   java -jar morphe-cli-1.9.1-dev.3-all.jar patch -p pairip-patches.mpp -e "Remove pairip protection" -e "Restore extracted methods" --bytecode-mode FULL --verify-with-sdk -o patched.apk twitter.apk 2>&1 | tee morphe-verify-log.txt

2. • Original installation package (this must be used for testing) https://wormhole.app/R85Z8a#XI-4IGEn9y8jiVTiFof1aA

3. Using this command in the CLI resulted in no output. Upon analysis, it was discovered that some classes had been accidentally deleted — an issue that the manager does not have.

morphe-verify-log.txt

3. This is the program that the manager has patched.
   https://wormhole.app/8K8BJP#Gal1qOJWu93qjhBT56tQzA
   (The problem is that the classes17.dex file has not been cleared or deleted.)

4. Regarding the duplicate class problem, it is caused by patching X-piko with the manager first, and then applying the PairIP-patches

The above response was generated using a translation tool and may contain errors or awkward phrasing.

[sjshb57]

My reply is a bit disorganized—sorry about that.

[sjshb57]

I was wondering if the issue of duplicate classes appearing after deleting several class in full mode can be fixed.
@wchill

[sjshb57]

It seems to be because of this
Tracking smali issue #100

Root Cause: DexPool Segment Overflow Rollback Mechanism
The chain of events is as follows (I have verified each step):
classMap is a Map<String, ClassDefWrapper> where the key is the type—there can be no duplicate types, and the input to compileFull is clean.
dexlib2’s internClass only stores the type ID for referenced classes, without the associated intern definition—multi-threaded parallel processing inherently prevents duplicates.
Therefore, duplicates can only originate from the overflow rollback logic added to DexPool by the Morphe fork’s smali (com.github.MorpheApp.smali): mark(), reset(), and hasOverflowed().
BasePool.reset() relies on a LinkedHashMap and tracks markedItemCount(size); during an overflow, it removes entries added after the mark from the end of the map. In FULL mode, when writing 200,000 classes—which cannot fit into a single DEX file (65,000-class limit)—the cycle of "intern → overflow → reset rollback → create new DEX" is inevitably triggered repeatedly. It is precisely during this rollback at the overflow boundary that certain classes are not fully rolled back or are written repeatedly to adjacent DEX files—which explains why the duplicate classes form a contiguous batch (com.x.dm.root.a0~z2) and are concentrated in the last two DEX files (the overflow boundary), whereas the original APK, which did not undergo this recompilation, does not exhibit this behavior.
Furthermore, Morphe included a comment in its dependency configuration stating "Tracking smali issue #100"—indicating that they are aware of this overflow/rollback-related bug, are tracking it, and have not yet fixed it.