This paper proposed new hybrid, analog-digital,
beamforming for a multiuser millimeter wave (mm-wave) relay
system. For this system, we consider a sum rate maximization
problem. The proposed hybrid beamforming is designed indirectly by considering a sum mean square error (MSE) minimization problem while utilizing the solution of digital beamforming.
To this end, we assume that the digital beamforming utilizes
the well known block diagonalization (BD) approach. Under this
assumption, we solve our problem as follows: First, we formulate
the sum rate maximization problem as the minimization of the
MSE between the received signal of the hybrid and digital beamforming designs.