On the Generative Nature of Prediction

We propose a notion of predictive models that extends the corresponding notion generally used in computational mechanics. We show that this extension allows for hidden Markov models that are more concise than the corresponding epsilon-machines. Thereby it provides more consistency with related theories such as Jaeger's theory of observable operator models. Furthermore, epsilon-machines turn out to be minimal within the usual context of predictive models.