In this paper, we introduce a novel framework which applies known image features combined with advanced linear image representations for weed recognition. Our proposed weed recognition framework, is based on state-of-the-the art object/image categorization methods exploiting enhanced performance using advanced encoding and machine learning algorithms. The resulting system can be applied in a variety of environments, plantation or weed types. This results in a novel and generic weed control approach, that in our knowledge is unique among weed recognition methods and systems. For the experimental evaluation of our system, we introduce a challenging image dataset for weed recognition. We experimentally show that the proposed system achieves significant performance improvements in weed recognition in comparison with other known methods.