Brainvisa Database

[katz]

Hello,

I have to write some process which have to be used both with the GUI and also with a shell. So I need to access the database, to keep the files organization, so I would like to know which function is used during this action to add files without updating the whole database when a user run the process in shell mode

Thanks for your help

Philippe Katz

[Dominique Geffroy]

Hi Philippe,

The easiest way is to list the ouput of your process in its signature. Indeed, all output item described in this signature will be automatically added to the database, provided there is a matching hierarchy rule for this type of data.
I advise you read the [urlhttp://brainvisa.info/doc/brainvisa-4.0/Programming_with_brainvisa.pdf]programming with Brainvisa tutorial[/url], especially the part about Database and Ontology if you need to define your own types of data and the matching onotlogy rules.

If your process creates many files and you don't want to put all them in the signature, it is possible to update only one directory of the database. Here is the code to do that:
from neuroHierarchy import databases
db=databases.database(<path to your database>)
db.update(directoriesToScan=<directory you want to update>)

Dominique

[katz]

Hi,

But if you create a signature, the process will be only run with the GUI, isn't it ? And I need to keep the possibility for my main classes to be run with the shell. So I think that the second solution fits better. But I have another problem that I don't understand .
When I update a directory :
db=databases.database( '/home/katz/brainvisa/share/shfj-4.0' )
db.update( '/home/katz/brainvisa/share/shfj-4.0/protocole_5/wallace/session_20080306/vsdi_data/exp03/trial016' )

I have the following error :
DatabaseError Traceback (most recent call last)

/home/katz/brainvisa/brainvisa/toolboxes/brainmonkey/processes/<ipython console> in <module>()

/home/katz/brainvisa/python/brainvisa/data/sqlFSODatabase.py in update(self, directoriesToScan, recursion, context)
421 context.write( self.name + ': parse directories and insert items' )
422 t0 = time.time()
--> 423 self.insertDiskItems( ( i for i in self.scanDatabaseDirectories( directoriesToScan=directoriesToScan, recursion=recursion ) if i.type is not None ), update=True )
424 duration = time.time() - t0
425 cursor = self._getDatabaseCursor()

/home/katz/brainvisa/python/brainvisa/data/sqlFSODatabase.py in insertDiskItems(self, diskItems, update)
689 cursor.executemany( 'INSERT INTO _FILENAMES_ (filename, _uuid) VALUES (? ,?)', (( relative_path(i, self.directory), uuid ) for i in diskItem.fullPaths()) )
690 except sqlite3.IntegrityError, e:
--> 691 raise DatabaseError( unicode(e)+': file names = ' + repr(diskItem.fullPaths()) )
692
693 values = [ uuid, format, os.path.basename( diskItem.fullPath() ) ]

DatabaseError: column filename is not unique: file names = ['/home/katz/brainvisa/share/shfj-4.0/protocol_4/wallace/session_20080306/vsdi_data/exp02/trial030/TC06_080306_E02B030.nii']

It is weird, because I have nothing else in than the file 'TC06_080306_E02B030.nii' in this directory and file names does not correspond to the directory I am updating but to a directory I tried to update before.

Thanks for your help

Philippe Katz

[riviere]

Hi Philippe,

No, using a signature doesn't means that the process will necessarily need a GUI: the GUI is just a view of a process signature, but the underlying process can still run without it (in normal processes at least). You can run a process from a python script (provided the BrainVisa environment is loaded) using the following syntax:

Code: Select all

context.runProcess( 'AimsConverter', read='/home/raymond/volume.ima', write='/home/raymond/othervolume.nii.gz' )

The context object is the one you receive as argument in a process execution() method. This context is normally the only link to the GUI or non-GUI execution mode (it is specialized for GUI but there are non-GUI contexts, which typically print messages in the terminal rather than in the process window)
If you are not in a process execution, there is a default non-GUI context (in the neuroProcesses module I think): defaultContext().

For the database error, we sometimes run into this kind of error in a non-reproducible way, for a reason that hasn't been discovered yet: we believe there is a bug somewhere linked to multithreading and concurrent access to the sqlite database. I don't know if you are in the same case or not. Basically it means that a given filename is inserted several times in the database (possibly with different uuids), which should not happen since they are tested before insertion...
Does it happen systematically for you ? If yes, there should be a way to fix it quite quickly, but I am perhaps not the best person to answer to that: Dominique and Yann know this part a bit better than I do, but they are both on holiday this week.

Denis

[Dominique Geffroy]

Hi Philippe,

Indeed, as Denis said, we are interested if you have a way to reproduce this error. We have already seen it but never systematically so we never understood why it occurs.

Dominique

[katz]

Sorry I was very long to answer.
So, I have found something to be sure to have this error.
When you update a WriteDiskItem which is already declared in the signature with:

Code: Select all

db=databases.database( database )
db.update( path )

I have also this error sometimes when I update with this code something which is not declared in the signature.

Philippe Katz

[Dominique Geffroy]

Hi Philippe,

Thanks for the clue. We will investigate about this problem.

But be carefull, when you want to update only a part of the database, you can indeed call db.update with the parameter directoriesToScan but it must be a list of paths, not just a path :
db.update( directoriesToScan=[path1, path2...] )
If you don't give a list, all the database is updated, that's why the error you got was about a file in another directory of the database.

Anyway, the error shouldn't occur, so there's probably a problem with diskitems uuid recording.

Dominique