[Feature Request] Cancelling search for any reason should return partial, even if incomplete signature instead of nothing, even if it isn't unique

[OshidaBCF]

The search has been going for 10 minutes now, and it's at least 200 byte long, while i don't mind waiting, when i inevitably decide to cancel it, i'd like to get a "partial" signature

if i'm lucky the signature will actually be unique, or will have only a couple results, easy to test

well while i was writing the search finished, and the signature is a whole 250 bytes long, but at least it's unique.

Btw this little signature is saving me a ton of time, i was first doing diff and after running for 2 hours it still wasn't done, this is gonna be so much faster

[mahmoudimus]

@OshidaBCF Heya, so I understand what you're proposing, if a search for a unique signature is going too long, if you decide to cancel it, dump previously tried signatures that narrowed down the candidates even if they aren't unique. Is that a good re-phrasing of your suggestion?

[OshidaBCF]

Yes

i don't know how the process works, but i assumed that the plugin just start where you tell it to start, then copy bytecode while going though subroutine calls ect, and each time it adds a couple bytes it does a search to see if it gets 1 result, and only when it reach 1 unique result does it output it

so i thought that even if you tell it to cancel a search, it already has a partially completed match, and it could output that one when cancelling

so yeah good refrasing

[mahmoudimus]

@OshidaBCF heya, took a swing at this. PR is up at #25.

short version: there's now a new opt-in checkbox on the main sigmaker form, "Output partial signature on cancel (opt-in)". default is off so nothing changes for anyone who doesn't ask for it. tick it, and when you cancel a unique-signature search mid-run you get something like:

Partial signature (NOT unique, 47 matches) for 0x140001234: E8 ?? ?? ?? ?? 45 33 F6 ...

the match count is the most recent number of database hits for the partial-so-far, which the loop was already computing every byte anyway and just throwing away. so no extra search cost on cancel, you just get the number that was about to be computed next.

a couple of intentional choices worth flagging:

• partials do not go to the clipboard, only to the output window. that way an accidental cancel can't clobber whatever you already had copied.
• if you cancel before any bytes have been appended (basically instant cancel) you still just get Operation cancelled by user -- no zero-byte garbage.

to try it:

1. fetch the new branch:

   git fetch origin
   git checkout fix/issue-22-partial-signature-on-cancel

2. copy src/sigmaker/__init__.py to your IDA plugins dir as sigmaker.py (overwriting the existing one). if you don't remember where that is, in IDA's Python console run: import idaapi, os; print(os.path.join(idaapi.get_user_idadir(), "plugins"))
3. restart IDA
4. hit Ctrl+Alt+S, tick the new "Output partial signature on cancel (opt-in)" checkbox, kick off a unique-sig search you know would normally take a while, and click Cancel mid-run

would love a 👍 or 👎 on whether this matches what you wanted. if the format of the partial line should look different, or if you'd rather have the clipboard write happen too, let me know and i'll adjust.

[OshidaBCF]

Just tested

Seems to be working, only problem i can find is the cancelled result always says "0 matches", shouldn't it say something >1, like "20 matches"

(in those two lines, it should say for exemple 1228 matches for the first, and 2 matches for the second)

Another bug that i don't think is caused by sigmaker, whenever a search is started, the whole thread of Ida is taken, so it "think" it's frozen, and clicking on cancel sometime take a bit to actually cancel, it also does not give me the regular "search has been going for 10s, do you want to continue, it might also be making sigmaker slower, because it seems to just freeze every 5 seconds for 1 seconds, i already had that before updating, and even before installing sigmaker, so probably just a weird setup on my side

[mahmoudimus]

@OshidaBCF thanks for testing! actually #24 wanted the exact opposite behavior here, so I gated it behind a flag. Try setting the config to 10 and retrying it and it should tell you about the "search has been going for 10s, do you want to continue?".

[mahmoudimus]

Another bug that i don't think is caused by sigmaker, whenever a search is started, the whole thread of Ida is taken, so it "think" it's frozen, and clicking on cancel sometime take a bit to actually cancel ...

I don't think this is a bug, this is intended behavior. I think it might be more interseting to fire off a thread and see if the search can go and work. I built a tool that does this already: https://github.com/mahmoudimus/ida-taskr

I can try to see if that's another enhancement.

[mahmoudimus]

@OshidaBCF nice catch, fix pushed to PR #25 (e4f4fd4 "fix: don't trust count_matches result when interrupted mid-scan").

quick diagnosis: when you click Cancel mid-search, IDA's wait-box cancel flag flips to True. our match-counting walks the binary internally and ALSO checks that same flag, so it bails out early and returns whatever partial count it had at the moment of cancel, which is usually 0. PR #25 was naively recording that 0 as the "match count" and printing it as the answer. the actual count was probably the previous iteration's number (in your screenshots, 1228 and 2 sound right).

new behavior: when the counter bails because of cancel, we now show match count unavailable instead of 0 matches. it's less informative than the real count but at least it's honest.

if you can pull the branch again and re-test (git checkout fix/issue-22-partial-signature-on-cancel && git pull), it'd be great to confirm.

re your other observation about the IDA UI freezing during the search: yeah, that's a single-threaded IDA limitation we don't easily work around. our wait box is supposed to let you cancel, and clicking Cancel does work, but the GUI between cancel clicks won't repaint because IDA's main thread is busy doing the search. we could add a process_events() yield every N instructions to keep the GUI breathing, but that's a separate change. open a new issue if you want to track that explicitly and i'll look at it after this PR lands.

also re the "search has been going for 10s, do you want to continue" prompt: that was the old behavior from before #18. we replaced it with the wait-box + cancel button by default. you can opt back in via the "Enable continue prompt (opt-in)" checkbox on the main form if you miss the periodic prompts.

[OshidaBCF]

Well #24 Is a good thing too, by default being a wait box is much better than the "Continue?" prompt

when i used the script, i turned off the Continue? prompt, as i knew it would be long, so i just wanted to let it run for 5-10 minutes

it's the "Has been running for 100 seconds" box gave me the idea, because i pressed cancel there and it just returned nothing which i though was a shame

I'll test the new one in a bit and i'll inform ya

[mahmoudimus]

@OshidaBCF guessing it works for you?

[OshidaBCF]

I think i forgot to test it, lemme do that really quick

After letting it run for couple minutes then cancelling it did outputted the first 67 pair of hexa (out of 262)

So at least that works no problem

but when the script runs, to "know" if it's unique, you just do an hexa search right ?

Couldn't you keep the number of result in memory and write it in the console when cancelling ?

Partial signature (NOT unique, 2 matches found)

This could also be shown on the window with the cancel button, along the cuz signature length, just to give users some feedback that shows that the script is digging ?

"Current signature length : 67 bytes"
"Current match count : 2"

I might be assuming wrong however.

[mahmoudimus]

@OshidaBCF good call, you're right. my previous fix was too cautious -- when count_matches got interrupted i threw away the unreliable in-flight count AND the previous iteration's trustworthy count along with it, which gave you "match count unavailable" when we actually had a perfectly good number sitting in memory from the iteration before.

pushed 4c36e3b to the same PR branch (#25). new behavior: when count_matches is interrupted mid-scan, keep last_match_count at the value from the last fully-completed iteration. so your example would now print Partial signature (NOT unique, 2 matches found) (or whatever the count was just before you clicked Cancel) instead of match count unavailable.

one small caveat worth flagging: the count corresponds to a signature one instruction SHORTER than the partial we emit (because we appended a byte just before the interrupted count_matches call). adding bytes can only narrow the match count, so the number we show is an upper bound on the actual matches for the partial. in practice it's usually accurate or within 1, but if you ever see "2 matches" and then search the partial and find only 1 match, that's why.

re your wait-box-feedback idea (showing current length + match count live in the cancel dialog): that's a separate change because the wait box message is static right now, and wiring live updates means hooking the generator's progress events into IDA's replace_wait_box. happy to do it as a follow-up PR if you want, just open a new issue describing what info you'd find most useful in the dialog and i'll scope it from there.

if you can re-test by pulling the branch again, much appreciated.