Batch Import Performance with Grails and MySQL
I found on Burt Beckwith’s blog, that there are 2 separate leaks, one of them is in the hibernate first-level cache, the other is a map that Grails uses for domain object validation errors.
Normally, a grails hibernate session executes something quickly and returns. During importing, we do a ton of processing, all with the same hibernate session. All of these objects that would normally be garbage collected when the session closed are piling up.The easiest way to deal with this is to create a simple method to clear out these collections periodically.We can modify our BookService to clean up GORM after every 100 books we insert:
class BookService {
def sessionFactory
def propertyInstanceMap = org.codehaus.groovy.grails.plugins.DomainClassGrailsPlugin.PROPERTY_INSTANCE_MAP
def importBooksInLibrary(library) {
library.eachWithIndex { Map bookValueMap, index ->
updateOrInsertBook(bookValueMap)
if (index % 100 == 0) cleanUpGorm()
}
}
def cleanUpGorm() {
def session = sessionFactory.currentSession
session.flush()
session.clear()
propertyInstanceMap.get().clear()
}
def updateOrInsertBook(Map bookValueMap) {
// ... same as above
}
}
http://naleid.com/blog/2009/10/01/batch-import-performance-with-grails-and-my...
