Researchers from the Department of Computer Science at Bar-Ilan University and NVIDIA’s AI research center in Israel have unveiled a groundbreaking method that significantly enhances how artificial intelligence models interpret spatial instructions when generating images. This advancement does not require retraining or altering the models themselves, marking a significant leap in AI technology.
Image-generation systems have long struggled with interpreting simple prompts such as “a cat under the table” or “a chair to the right of the table,” often misplacing objects or ignoring spatial relationships entirely. The innovative solution from the Bar-Ilan research team enables AI models to adhere to such instructions with improved accuracy in real time.
Introducing the Learn-to-Steer Method
The newly developed method, named Learn-to-Steer, functions by examining the internal attention patterns of an image-generation model. This approach provides insights into how the model organizes objects spatially. A lightweight classifier then subtly guides the model’s internal processes during image creation, ensuring more precise placement of objects as per user instructions. Notably, this method can be integrated with any existing trained model, thereby eliminating the need for costly retraining.
In the Stable Diffusion SD2.1 model, accuracy in understanding spatial relationships increased from 7% to 54%. In the Flux.1 model, success rates improved from 20% to 61%, with no adverse impact on the models’ overall capabilities.
Expert Insights and Implications
“Modern image-generation models can create stunning visuals, but they still struggle with basic spatial understanding,” noted Prof. Gal Chechik from the Department of Computer Science at Bar-Ilan University and NVIDIA. “Our method helps models follow spatial instructions more accurately while preserving their general performance.”
Sapir Yiflach, the study’s lead researcher and co-author alongside Prof. Chechik and Dr. Yuval Atzmon of NVIDIA, elaborated: “Instead of assuming we know how the model should think, we allowed it to teach us. This enabled us to guide its reasoning in real time, essentially reading and steering the model’s thought patterns to produce more accurate results.”
Potential Applications and Future Prospects
The findings from this research open new avenues for enhancing the controllability and reliability of AI-generated visual content. Potential applications span across various fields, including design, education, entertainment, and human-computer interaction. As AI continues to permeate these industries, the ability to accurately follow spatial instructions could significantly enhance user experience and output quality.
This development follows a growing trend in AI research focusing on improving the interpretative capabilities of models without extensive retraining. By refining how AI systems understand and execute spatial instructions, researchers are paving the way for more intuitive and versatile AI applications.
The research is set to be presented in March at the WACV 2026 Conference in Tucson, Arizona, where it is expected to garner significant attention from the AI community. As the industry continues to evolve, such advancements underscore the importance of innovative approaches in overcoming existing limitations and expanding the potential of AI technologies.
