The importance of discovering significant variables from a large candidate pool is now widely recognized in many fields. There exist a number of algorithms for variable selection in the literature. Some are computationally efficient but only provide a necessary condition for, not a sufficient and necessary condition for, testing whether a variable contributes or not to the system output. The others are computationally expensive. The goal of the paper is to develop a directional variable selection algorithm that performs similar to or better than the leading algorithms for variable selection, but under weaker technical assumptions and with a much reduced computational complexity. It provides a necessary and sufficient condition for testing whether a variable contributes or not to the system. In addition, since indicators for redundant variables aren’t exact zeros, it is difficult to decide whether variables are redundant or not when the indicators are small.To solve this problem, a penalty optimization algorithm is proposed to ensure the convergence of the set. Simulation and experimental results verify the effectiveness of the directional variable selection method proposed in this paper.