bugs in foundry test generation

[monperrus]

Here, I collect bugs in foundry test generation

contract name

Error (2333): Identifier already declared.
 --> test/Test.1904265591758374514.sol:6:1:
  |
6 | contract Test is Test {

should be for example Test1 is Test

fallback

call to fallback function is written as an empty string

Error (2314): Expected identifier but got '('
  --> test/Test.8771174626403390381.sol:19:16:
   |
19 |         Target.();

null byte

files contains \0 (written as below)

Error (8936): Expected even number of hex-nibbles.
  --> test/Test.2952294588457979761.sol:23:29:
   |
23 |         Target.renounceRole(hex"<nul><nul><nul>", address(0xa329c0648769a73afac7f9381e08fb43dbea72));
   |  

[gustavo-grieco]

Great report! You can try using the sub issue GitHub feature to keep them in a list

[gustavo-grieco]

You can also ask ChatGPT/Claude/Gemini to create PRs for each one, as they are capable to do it.

[BowTiedRadone]

Hi @monperrus. I will give this a shot, if you have anything that would help reproduce it would be highly appreciated 🙏

[monperrus]

Hi @BowTiedRadone!

Awesome.

contract bugfallback {
  event AssertionFailed();
  function f() external {} // workaround ABI is empty, are you sure your constructor is right?
  fallback() external {
    emit AssertionFailed();
  }
}

$ ./echidna --test-mode assertion --corpus-dir foobar --format text bugfallback.sol

For some reason, no forge test is written with this.

[gustavo-grieco]

"assertion failed" tests are not supported. For the first step, we will omit that case, since I think it is a bit harder to fix

[monperrus]

@gustavo-grieco I'm puzzled. you told me in #1462 (comment) that " Foundry generation only works in assertion mode" and I indeed go some.

[gustavo-grieco]

My mistake, the assertion mode using events is not supported yet. 😅

[BowTiedRadone]

@monperrus Picking up this issue now, trying to fix the following days. For the contract name bug, I don't necessarily think we want a dynamic approach (Test<suffix> if contract name is Test). The naming collision happens because of the forge-std/Test.sol import (and then the test contract extending a contract with the same name). This means anything that is not reserved other than Test would work here.

@gustavo-grieco What about FoundryTest (EchidnaTest would look misleading)?

[monperrus]

This means anything that is not reserved other than Test would work here.

yes!

thanks @BowTiedRadone

[gustavo-grieco]

@BowTiedRadone maybe it will be good to fix the other issues in the same PR you are working right now?

[BowTiedRadone]

@gustavo-grieco Sure, as long as #1512 won't get too verbose (it shouldn't).