added kafka integration

[jensendw]

@truthbk Added kafka integration + tests, followed the same pattern as with mongo integration so should be relatively solid.

[truthbk]

@jensendw with spec tests and all! This looks very good already. I'll give it a closer review, and should get it merged soon! Thank you so much! :)

[jensendw]

@truthbk Any news on when this will be merged and released? I was hoping to stop using my forked module.

[truthbk]

@jensendw I'm sorry this got left behind. There were two kafka PRs, quite similar, but actually this was probably more feature rich and the other one got merged instead. I am going to reintroduce this one for the 2.0 module we are preparing to enable the multi-instance support you implemented. Thank you and apologies.

[jensendw]

No worries, thanks for letting me know

[truthbk]

After merging #387, going to merge this now - additional work will be conducted in a separate PR to make the manifest backward compatible as Kafka is a very important integration for many of our customers and we'd like to make the "upgrade" seamless fo them. I will merge this now once the CI tests run 💚

[truthbk]

Looking great! Thank you @jensendw, I will make some small changes to your work to make the manifest backward-compatible, but this is awesome! 🙇