Quite often, computing the exact result of a problem becomes very resource-intensive and an approximation to it suffices for most purposes. In an “Approximate Computing” paradigm, we aim to save both time and energy by providing a “good enough” solution. Furthermore, modern circuits often exhibit manufacturing variations that can never guarantee computations with very high precision. This gives rise to the concept of Stochastic Computing which can replace large conventional computation units with small logic circuits. In this work we investigate how Magnetic Tunnel Junctions can be used as Stochastic Number Generators and how their properties can be exploited to design an energy-efficient Neural Network architecture. Please stay tuned for our contributions in this new and upcoming area.