Biochemical systems are robust in nature. We define robustness of a biochemical system as the property where during time evolution, a closed system returns to its initial state. In this study, we propose some mathematical formulations to analyse the robustness of a closed biochemical system. We have provided a tentative guideline towards applying the theory to a non-closed system. We know that a biochemical system evolves with time as a continuous-time Markov process. When this Markov chain is irreducible, it can be proved theoretically that the system will always return to its initial state, and also the expected time of return can be determined. This return time depends upon the stationary probability distribution, which is determined as the solution of an eigenvalue equation $xQ=0$ where $Q$ is the transition rate matrix. We calculate this expected return time for five different closed systems: unidirectional cyclic linear chains, bidirectional cyclic linear chains and three real biological systems, and verify the theoretical results against the average return time obtained by stochastic simulation.