Chrono Restart Issues
Hi,
I am running a simulation using Chrono (collision + linear spring). As with the latest update, we are able to restart simulations using Chrono.
Initially, I restarted my simulation after the first time step and was successfully able to restart it. I ran a new simulation for 48 hours. It was stopped after 48 hours due to job runtime limitations with around 326 timesteps.
When i tried restarting with timestep 326, I am encountering the following error
I looked at the ErrorBoundaryOut file - it contains no data
export name=IrregularWaveTestA1 export dirout=${name}_out export diroutdata=${dirout}/data # "executables" are renamed and called from their directory export dirbin=..../bin/linux export LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:${dirbin} export gencase="${dirbin}/GenCase_linux64" export dualsphysicscpu="${dirbin}/DualSPHysics5.0CPU_linux64" export dualsphysicsgpu="${dirbin}/DualSPHysics5.0_linux64" export boundaryvtk="${dirbin}/BoundaryVTK_linux64" export partvtk="${dirbin}/PartVTK_linux64" export partvtkout="${dirbin}/PartVTKOut_linux64" export measuretool="${dirbin}/MeasureTool_linux64" export computeforces="${dirbin}/ComputeForces_linux64" export isosurface="${dirbin}/IsoSurface_linux64" export flowtool="${dirbin}/FlowTool_linux64" export floatinginfo="${dirbin}/FloatingInfo_linux64" export olddiroutdata=${diroutdata} export dirout=${name}_Part2_out export diroutdata=${dirout}/data ${gencase} ${name}_Def ${dirout}/${name} -save:all if [ $? -ne 0 ] ; then fail; fi ## ## Executes DualSPHysics to simulate SPH method. ${dualsphysicsgpu} -gpu ${dirout}/${name} ${dirout} -dirdataout data -svres -restartchrono:1 -partbegin:0325 ${olddiroutdata} if [ $? -ne 0 ] ; then fail; fi
This is my Linux Job File used for restarting. If you could let me know what issue could be the reason, I can restart my simulation
Debug Trace
Notice |
rich is deprecated. Use FormatService::renderHtml($content, Formats\RichFormat::FORMAT_KEY) instead. #0 [internal function]: gdn_ErrorHandler(16384, 'rich is depreca...', '/var/www/forums...', 950, Array) #1 /var/www/forums-dual-sphysics-org/library/core/functions.general.php(950): trigger_error('rich is depreca...', 16384) #2 /var/www/forums-dual-sphysics-org/library/core/class.format.php(1729): deprecated('rich', 'FormatService::...') #3 /var/www/forums-dual-sphysics-org/library/core/class.format.php(1479): Gdn_Format::rich('[{"insert":"Hi,...') #4 /var/www/forums-dual-sphysics-org/applications/vanilla/controllers/class.discussioncontroller.php(197): Gdn_Format::to('[{"insert":"Hi,...', 'Rich') #5 /var/www/forums-dual-sphysics-org/library/core/class.dispatcher.php(862): DiscussionController->index('2489', 'chrono-restart-...', '') #6 /var/www/forums-dual-sphysics-org/library/core/class.dispatcher.php(279): Gdn_Dispatcher->dispatchController(Object(Gdn_Request), Array) #7 /var/www/forums-dual-sphysics-org/index.php(29): Gdn_Dispatcher->dispatch() #8 {main} |
Notice |
rich is deprecated. Use FormatService::renderHtml($content, Formats\RichFormat::FORMAT_KEY) instead. #0 [internal function]: gdn_ErrorHandler(16384, 'rich is depreca...', '/var/www/forums...', 950, Array) #1 /var/www/forums-dual-sphysics-org/library/core/functions.general.php(950): trigger_error('rich is depreca...', 16384) #2 /var/www/forums-dual-sphysics-org/library/core/class.format.php(1729): deprecated('rich', 'FormatService::...') #3 /var/www/forums-dual-sphysics-org/library/core/class.format.php(1479): Gdn_Format::rich('[{"insert":"Hi,...') #4 /var/www/forums-dual-sphysics-org/applications/vanilla/views/discussion/helper_functions.php(24): Gdn_Format::to('[{"insert":"Hi,...', 'Rich') #5 /var/www/forums-dual-sphysics-org/applications/vanilla/views/discussion/discussion.php(89): formatBody(Object(stdClass)) #6 /var/www/forums-dual-sphysics-org/applications/vanilla/views/discussion/index.php(31): include('/var/www/forums...') #7 /var/www/forums-dual-sphysics-org/library/core/class.controller.php(778): include('/var/www/forums...') #8 /var/www/forums-dual-sphysics-org/library/core/class.controller.php(1382): Gdn_Controller->fetchView('', false, false) #9 /var/www/forums-dual-sphysics-org/library/core/class.pluggable.php(217): Gdn_Controller->xRender() #10 /var/www/forums-dual-sphysics-org/applications/vanilla/controllers/class.discussioncontroller.php(310): Gdn_Pluggable->__call('render', Array) #11 /var/www/forums-dual-sphysics-org/library/core/class.dispatcher.php(862): DiscussionController->index('2489', 'chrono-restart-...', '') #12 /var/www/forums-dual-sphysics-org/library/core/class.dispatcher.php(279): Gdn_Dispatcher->dispatchController(Object(Gdn_Request), Array) #13 /var/www/forums-dual-sphysics-org/index.php(29): Gdn_Dispatcher->dispatch() #14 {main} |
Notice |
rich is deprecated. Use FormatService::renderHtml($content, Formats\RichFormat::FORMAT_KEY) instead. #0 [internal function]: gdn_ErrorHandler(16384, 'rich is depreca...', '/var/www/forums...', 950, Array) #1 /var/www/forums-dual-sphysics-org/library/core/functions.general.php(950): trigger_error('rich is depreca...', 16384) #2 /var/www/forums-dual-sphysics-org/library/core/class.format.php(1729): deprecated('rich', 'FormatService::...') #3 /var/www/forums-dual-sphysics-org/library/core/class.format.php(1479): Gdn_Format::rich('[{"insert":"I b...') #4 /var/www/forums-dual-sphysics-org/applications/vanilla/views/discussion/helper_functions.php(24): Gdn_Format::to('[{"insert":"I b...', 'Rich') #5 /var/www/forums-dual-sphysics-org/applications/vanilla/views/discussion/helper_functions.php(170): formatBody(Object(stdClass)) #6 /var/www/forums-dual-sphysics-org/applications/vanilla/views/discussion/comments.php(19): writeComment(Object(stdClass), Object(DiscussionController), Object(Gdn_Session), 1) #7 /var/www/forums-dual-sphysics-org/applications/vanilla/views/discussion/index.php(53): include('/var/www/forums...') #8 /var/www/forums-dual-sphysics-org/library/core/class.controller.php(778): include('/var/www/forums...') #9 /var/www/forums-dual-sphysics-org/library/core/class.controller.php(1382): Gdn_Controller->fetchView('', false, false) #10 /var/www/forums-dual-sphysics-org/library/core/class.pluggable.php(217): Gdn_Controller->xRender() #11 /var/www/forums-dual-sphysics-org/applications/vanilla/controllers/class.discussioncontroller.php(310): Gdn_Pluggable->__call('render', Array) #12 /var/www/forums-dual-sphysics-org/library/core/class.dispatcher.php(862): DiscussionController->index('2489', 'chrono-restart-...', '') #13 /var/www/forums-dual-sphysics-org/library/core/class.dispatcher.php(279): Gdn_Dispatcher->dispatchController(Object(Gdn_Request), Array) #14 /var/www/forums-dual-sphysics-org/index.php(29): Gdn_Dispatcher->dispatch() #15 {main} |
Comments
I believe the issue would be the third warning, that it is not fully supported yet. Perhaps what happened was that when it tried to re-initialize from the latest time step, somewhere in the code it mistakenly ended up dividing by zero and producing nan results.
Nan results are of course out of bounds and breaks the restart.
This is just me theorizing, I haven't played with the latest update yet, waiting for the new release next year hopefully! :-)
I hope you are able to get it working!
Kind regards