Selenium (Se) is a widely distributed trace element with dual (beneficial or toxic) effects for humans, animals, and plants. The availability of Se in the soil is reliant on the structure of the parental material and the procedures succeeding to soil formation. Anthropogenic activities affect the content of Se in the environment. Although plants are the core source of Se in animal and human diet, the role of Se in plants is still debatable. A low concentration of Se can be beneficial for plant growth, development, and ecophysiology both under optimum and unfavorable environmental conditions. However, excess Se results in toxic effects, especially in Se sensitive plants, due to changing structure and function of proteins and induce oxidative/nitrosative stress, which disrupts several metabolic processes. Contrary, Se hyperaccumulators absorb and tolerate exceedingly large amounts of Se, could be potentially used to remediate, i.e., remove, transfer, stabilize, and/or detoxify Se-contaminants in the soil and groundwater. Thereby, Se-hyperaccumulators can play a dynamic role in overcoming global problem Se-inadequacy and toxicity. However, the knowledge of Se uptake and metabolism is essential for the effective phytoremediation to remove this element. Moreover, selecting the most efficient species accumulating Se is crucial for successful phytoremediation of a particular Se-contaminated area. This review emphasizes Se toxicity in plants and the environment with regards to Se biogeochemistry and phytoremediation aspects. This review follows a critical approach and stimulates thought for future research avenues.