Question: Which software do you use for data analysis?
Hey guys!
This is more like an "open" question to see how others analyse their data, which they get from DualSPHysics simulations. So basically some questions I find interesting would be:
Kind regards
This is more like an "open" question to see how others analyse their data, which they get from DualSPHysics simulations. So basically some questions I find interesting would be:
- Do you use binary data directly or export to csv / vtk?
- Do you use spread sheet programs (Excel etc.) or something like Matlab/Python etc.?
- What is your biggest challenge with handling data? (Storage, reading speed etc. maybe something else?
Kind regards
Comments
2. I use mostly excel, since my .CSV files differ marginally each time. I could write a script, but deemed it not worthy (would not win any time).
3. Setting up the sampling of the raw data, so slightly changing the .sh (script) file which handles the inbuilt functions such as IsoSurfaces and MeasureTool. And mostly modifying the supporting files such as Fileboxes.txt or VelocityPoints.txt (sampling locations).
Excellent question !
1. I use .csv and .vtk with paraview for quick visualization and with blender for nice rendering.
2. I switch from excel to Python recently. It makes all the difference for me. I tend to make a few simulations with only one parameter changing (flow for exemple) so the script approach is very usefull to treat the data and then do some science with it. But even with a single simulation, I would go with Python, because it is a better and more reliable way to treat or explore data (IMO) (also if you want to modify your work you have a history and can easily do so).
3. Some measuretool export can be very long (more than an hour with i5-8600 (3.1 GHz))... Also, I still haven't managed to make my own export function for the binary data. I would want to add some visualization option...
Kind regards
For bladomas1
- For your answer to question 3., could you explain what you mean with sampling? Are you talking about exporting csv files for specific locations inside your simulation domain? Like velocity at point x,y,z?
- If this is the case then; is the problem finding the exact location to probe?l
For TPouzol- How does the Python script you made work? Does it read all the arrays you want from the csv file and then you do the math you want to do?
- You say you are interested in making your own export of a binary file, what is your problem regarding this right now?
Kind regards2. I'd say time and "fear". As I'm self taught in code, it still is a bit daunting to compile my tool. However, I intend to remedy this problem soon.
1. Exactly, setting up the specific locations for exporting data (I call it sampling). So for velocity I provide a list of points for data gathering (X,Y,Z). For measuretool boxes have to be drawn to gather the data (specify 8 points X Y Z).
2. I have often slightly different setups (simulations) which means I cannot gather the data using the old probe locations, and need to write new ones. This costs each time some time. I did consider writing a python script that reads the Def.xml file and creates the appropriate probe locations, but since my project scope is quite limited it was not worth the effort.
I would still agree with @TPouzol that Python is better, but only if you work on the project for a long time. Then using scripts can save you time. For me however, I wanted quick results and had varying simulations, so using Excel was optimal.
Kind regards