Improve callbacks

[torfjelde]

Callbacks can be incredibly useful, e.g. TuringCallbacks.jl with TensorBoardLogging.jl integration.

However, there are many "standard" callbacks that we arguably should provide some default implementation (i.e. that have "zero" dependencies) of:

• StateHistoryCallback: simply extracts the states from a sample call. This is useful if one wants to inspect sampler parameters, etc. that aren't necessarily part of the transitions. Ref: sample equivalent but including states #84
• : MultipleCallback: a wrapper around multiple callbacks to make it easier to pass many of them. Ref: Allow multiple callbacks #80
• ???

The question is then: should these go in AbstractMCMC.jl itself, or should we put them in a separate package?

And if we're putting them in a separate package (which I also think is the best approach here), then do we put them in a package called AbstractMCMCCallbacks.jl or in the existing TuringCallbacks.jl?

Thoughts? @devmotion @yebai @sunxd3 @mhauru @penelopeysm

[devmotion]

Could the TensorBoardCallback moved to an extension? Then TuringCallbacks would seem rather lightweight and adding more callbacks there would seem the most straightforward approach.

MultipleCallback

TuringCallbacks already contains MultiCallback, isn't that the same thing?

[torfjelde]

Could the TensorBoardCallback moved to an extension? Then TuringCallbacks would seem rather lightweight and adding more callbacks there would seem the most straightforward approach.

Yeah, could potentially do that. Though it's original intention was to be "batteries included". But yeah, maybe it makes more sense to just put it in an extension for now.

TuringCallbacks already contains MultiCallback, isn't that the same thing?

Yep yep, so it's more that TuringCallbacks.jl is somewhat of a heavy dep.