We encountered an interesting bug while working on our web interfaces. We posted it to Sun, but we are curious if the security community sees any way to exploit this in more than a DOS sense. This isnt our speciality, that's why we are inquiring here.
This is a copy of the post to Sun's bug tracking, posted 2006-01-09
A DESCRIPTION OF THE PROBLEM :
Running a simple script on a web page using Internet Explorer cause the IE GUI Handles to grow up to 10000. This behavior can be reproduced only when running Sun's JVM V1.5.0_06.
ERROR MESSAGES/STACK TRACES THAT OCCUR :
No error message. When application reaches over 10 000 GUI Handles it goes crazy. Windows flicking, resizing, moving. etc. Looks like either handles that arent free are being re-used, or there's a buffer overflow into the memory space of these 10k handles.
This bug can be reproduced.
In a web page, in IE6.
---------- BEGIN SOURCE ----------
---------- END SOURCE ----------
Just monitor GDI handles (with processExplorer for example)
We tested on XP SP2, and Win2k SP4, fully patched. Only version 1.5.0_06 (latest) of Sun's JVM exhibit this bug. Previous version appear to be ok. MashX discovered/isolated this bug. Much thanks.