History | Log In     View a printable version of the current page.  
Issue Details (XML | Word | Printable)

Key: COR-1068
Type: Bug Bug
Status: Closed Closed
Resolution: Fixed
Priority: Critical Critical
Assignee: Norberto Goussies
Reporter: Luciano Di Cocco
Votes: 0
Watchers: 2
Operations

If you were logged in you would be able to see more operations.
db4o Core

IExtObjectContainer.GetByID on invalid ID throws OutOfMemoryException and closes the database

Created: 05/Jan/08 11:09 PM   Updated: 22/Jul/08 08:39 PM
Component/s: None
Affects Version/s: 6.4.15, 7.0.22
Fix Version/s: None

Time Tracking:
Issue & Sub-Tasks
Issue Only
Original Estimate: Not Specified
Remaining Estimate: 0h
Time Spent - 7.67h
Time Spent: 7.67h
Time Spent - 7.67h

Environment: .NET

Peers: Adriano Verona
Order: 2
Iteration: 25
Original IDS Estimate: 1
Resolution Date: 15/Jan/08 02:37 PM
First Response Date: 08/Jan/08 12:51 PM
Labels:
Participants: Carl Rosenberger, Judah Gabriel, Luciano Di Cocco and Norberto Goussies
Number of Attachments: 0
Number of Comments: 6

Sub-Tasks  All   Open   

 Description  « Hide
IExtObjectContainer.GetByID in version 6.4 is supposed to throw InvalidIDException (the documentation states that it returns nell, but I never get that value).

I found that with some ID OutOfMemoryException is called and the database is closed. The values of ID that get this behavoiur seems to vary somehow from db to db, but I found that

IObjectContainer oc1 = Db4oFactory.OpenFile("database.yap");

OutOfMemoryException ex1 = null;

for (long i = 1; i < 100; i++)

{

try

{

object obj = oc1.Ext().GetByID(i);

}

catch (Db4objects.Db4o.Ext.InvalidIDException ex)

{

}

catch (OutOfMemoryException ex)

{

ex1 = ex;

break;

}

}

exits with the databse closed on i = 17 on all my test databases.
 
 
 I have done some tracing in the code to find the source of the problem. It seems there's something unexpected in the file structure. LocalObjectContainer.ReadReaderOrWriterByID calls LocalTransaction.GetCurrentSlotOfID to get the right slot.

ReadPointer seems to find something unexpected. It seems not to recognize that the address is invalid. If the length is reasonable the error is catched in ReadEncrypt as an IncompatibleFileSystem exception, but il the length is big there is an OutOfMemory and the database is closed.

I suspected the problem was that I used databases generated with 6.1, but I get the same result with newly generated databases, even without any deleted item.


 


 All   Comments   Work Log   Change History      Sort Order: Ascending order - Click to sort in descending order
Carl Rosenberger - 08/Jan/08 12:51 PM
[12:11:57] Patrick Roemer says: COR-1068: If you pass an ID into db4o, it will read 8 bytes from this address, interpret the last 4 bytes as a length specification and allocate a buffer of this size - if the pointer is bogus and the "length" is a few dozen terabytes: game over, OOME. As a quick measure, we could add a sanity threshold for the buffer length, but for a real solution, ID handling will have to be redesigned in general (by moving IDs to BTrees, or using specific marker bits for pointers, or...) , I guess.
[12:25:10] Carl Rosenberger says: Indeed Patrick.
[12:25:29] Carl Rosenberger says: Actually we could do a halfway sane sanity check too:
[12:25:51] Patrick Roemer says: Check whether the pointer address is inside the db file?
[12:25:58] Carl Rosenberger says: Yeah, that's a good one.
[12:26:01] Carl Rosenberger says: Second one:
[12:26:56] Carl Rosenberger says: If the length value appears to be large (50 MB or more), create a byte buffer with that length and catch OOME. On OOME throw something more useful.
[12:27:04] Carl Rosenberger says: InvalidIDException or so.
[12:27:30] Carl Rosenberger says: Catch InvalidIDException only in getByID calls. Bubble up and shut down engine everywhere else.
[12:28:17] Carl Rosenberger says: Rethrow InvalidIDException in getByID.

Judah Gabriel - 11/Apr/08 03:40 PM
Carl and friends,

This is a pretty serious issue for us. We utilize Lucene.NET and some other tools to keep track of IDs for lookup later. If the ID isn't in the database, we get either an OutOfMemoryException or an OverflowException ("arithmetic resulted in an overflow"). These both close the database, causing us a lot of headaches.

Is it possible to make GetByID always throw an InvalidIDException when the ID isn't in the database? And also not close the database? This would save us many headaches!

Thanks.

Judah Gabriel - 22/Jul/08 07:33 PM
Update: correction, we either get an OutOfMemoryException of an ArithmeticException when this happens.

This is very important to us to receive an InvalidIdException rather than an OutOfMemoryException that closes the database.

Judah Gabriel - 22/Jul/08 07:46 PM
Ah, I didn't see you guys resolved this issue as fixed. Thanks! We'll test it out.

Judah Gabriel - 22/Jul/08 08:22 PM
After testing, this seems to have fixed the bug. Thanks guys!

Judah Gabriel - 22/Jul/08 08:39 PM
Upon further testing, the OOME is indeed fixed, however, 2 other issues remain:

-the database gets closed when InvalidIDException is thrown
-sometimes an ArithmeticException is thrown instead of an InvalidIDException.

I've created sub-cases for these issues.