Optimizer Suggestions for Common Graph Fusions

[AyoubMDL]

I've been using ONNXScript extensively lately and really appreciate how powerful and flexible it is. As part of that experience, I and @Johansmm would like to suggest a few optimization patterns that could improve performance and simplify common model graphs. These are generally applicable across many models:

1. Fold BatchNormalization into preceding nodes (Conv, ConvTranspose, Gemm) Rewriter: Fold Batchnorm nodes #2312
2. Fuse MatMul + Add into Gemm [Rewriter]: Add ∘ MatMul -> Gemm #2356
3. Fuse Min + Max into Clip [Rewriter]: add fusion rules for successive Min/Max patterns #2500
4. Eliminate redundant Reshape / Flatten nodes [rewriter] Unify reshape flatten ops #2518
5. Fold Pad into Conv / ConvInteger [Rewriter] Add optimizer to fold Pad operators into Conv #2363
6. Fuse successive Clip / ReLU nodes [Rewriter]: fuse successive Relu/Clip nodes #2410

We're happy to contribute if any of these are a good fit. Feel free to add other suggestions or let us know if some of these are out of scope.

[justinchuby]

Thanks! All of the above sounds good. PRs welcomed. For (2) we want it to be optional because some runtimes prefer one way over another. You may add the rules in https://github.com/microsoft/onnxscript/tree/main/onnxscript/rewriter and use the existing ones as reference.