Is your feature request related to a problem? Please describe.

In some cases, we want to filter S3 objects when using S3/SQS as a source. For example, I may only want objects with the assets/ prefix.

The S3 scan supports filtering, but the options are nested under scan. So they can't be used with SQS.

Describe the solution you'd like

Move the filters options to the top-level and make them available for filtering regardless of the notification mechanism.

source:
  s3:
    filters:
      my-bucket1:
        include_prefix:
          - assets/
        exclude_suffix:
           - .jpg
           - .xml
      my-bucket2:
        include_prefix:
          - another/path/to/filter/

These filters can apply for both SQS and scan. We would not permit these filter along with normal scan filters. And as a future breaking change, we can remove the filters from the S3 scan options.

This approach is similar to how we handle bucket ownership. There is only one configuration and either SQS or scan use it.

Describe alternatives you've considered (Optional)

In many cases, we could filter on the S3 notifications that go to SQS. However, this is not as flexible as filtering in Data Prepper would be.

Additional context

N/A