Higher resolution, Linux scripts and RAM

cadff1cadff1
in DualSPHysics v5.0

Dear developers,

I have a problem when trying to use a higher resolution.

Until dp = 0.003 (modelling and tests) it was possible to run simulations in Windows.

However, I need to use higher resolutions in simulation. When trying to use the Linux scripts on a server, "xCaseDambreak_linux64_CPU" and "xCaseDambreak_linux64_GPU", the output directory is empty (without output files) and the simulation is aborting after some time. I thought that the problem was related to the RAM, but since I increase dp to test the scripts the output data is not being saved.

Do you have experience with this situation?

Regards.

Comments

  • AlexAlex
    edited July 2021

    Can you please paste here the output information? Case.out (output of GenCase) and Run.out (output of DualSPHysics)

    Regards

  • cadff1cadff1

    Dear Alex,

    Following the information (the data directory is empty):


    Case.out:

    GenCase v5.0.229 (29-01-2021) 

    ===============================

    OmpThreads: 2

    [LoadXMLInit]

    List: MAIN

    Total operations: 25

    [LoadXMLInit-End time:0.003s]

    [Draw]

    List of available variables: Gravity_x=[0] Gravity_y=[0] Gravity_z=[-9.81] Rhop0=[1000] Dp=[0.001] PosMin_x=[-1.724] PosMin_y=[-0.05] PosMin_z=[-0.4] PosMax_x=[0.924] PosMax_y=[0.41] PosMax_z=[0.3] CaseName=["CaseDambreak"] Data2D=[0] Data2DPosy=[0] H=[0.00173205] MassBound=[1e-06] MassFluid=[1e-06]


    Mk size: 1 bytes

    Case limits: (-1.724,-0.05,-0.4) - (0.924,0.41,0.3)

    Distance between points (Dp): 0.001

    Case domain: (0,0,0)-(2647,458,698)

    Domain points: 2648 x 459 x 699 = 849586968

    Case parts: 1 x 699z

    Memory: 0.79Gb 810.23Mb (849586968)


    > MAIN

    FileShapes> flume_Dp.vtk shapes:4 points:8  (time:0.002s)

    FileShapes> flume_Actual.vtk shapes:4 points:8  (time:0.002s)

    FileShapes> threshold_Dp.vtk shapes:12 points:8  (time:0.001s)

    FileShapes> threshold_Actual.vtk shapes:12 points:8  (time:0.002s)

    FileShapes> pool_Dp.vtk shapes:5 points:8  (time:0.002s)

    FileShapes> pool_Actual.vtk shapes:5 points:8  (time:0.001s)

    FileShapes> sides_pool_Dp.vtk shapes:3 points:8  (time:0.002s)

    FileShapes> sides_pool_Actual.vtk shapes:3 points:8  (time:0.001s)

    FileShapes> dam_Dp.vtk shapes:39813 points:23152  (time:0.005s)

    FileShapes> dam_Actual.vtk shapes:39824 points:26178  (time:0.004s)

    FileShapes> big_wedge_Dp.vtk shapes:13941 points:7304  (time:0.003s)

    FileShapes> big_wedge_Actual.vtk shapes:13944 points:9088  (time:0.003s)

    FileShapes> block_Dp.vtk shapes:12300 points:5297  (time:0.002s)

    FileShapes> block_Actual.vtk shapes:12428 points:6652  (time:0.002s)

    <MAIN (Operations= 17+8+7 = 32)


    Triangles: 66232 (142 ignored)

    [Draw-End time:2.657s]


    [LoadsXMLConfig]

    Execution parameters.

    Initials configuration.

    Floating bodies configuration.

    Properties configuration.

    Predefined motion configuration.

    [LoadsXMLConfig-End time:0.027s]


    [PreparePoints]

    Computing Vtk-Cells data... (time:8.392s)

    BitSizeTot: 33 (3 + 12+9+9)

    Points loaded: 86941999

    Sorting points by Z...

    Sorting points by Mk...


    List of Points:

    Points -> Fixed-Boundary particles:

     block[ 0] (mk: 10): 2029237 --> 2029237

     block[ 1] (mk: 11): 447780 --> 447780

     block[ 2] (mk: 12): 336375 --> 336375

     block[ 3] (mk: 60): 674043 --> 674043

     block[ 4] (mk: 65): 77076 -->  77076

     block[ 5] (mk: 75): 527216 --> 527216

    Points -> Floating-Boundary particles:

     block[ 6] (mk: 70): 42343 -->  42343

    Points -> Fluid particles:

     block[ 7] (mk: 1): 82807929 --> 82807929


    Particle summary:

     Fixed....: 4091727 id:(0-4091726)  MKs:6 (10-12,60,65,75)

     Moving...: 0

     Floating.: 42343 id:(4091727-4134069)  MKs:1 (70)

     Fluid....: 82807929 id:(4134070-86941998)  MKs:1 (1)


    Total particles: 86941999 (bound=4134070 (fx=4091727 mv=0 ft=42343) fluid=82807929)

    Total MK blocks: 8 (bound=7 (fx=6 mv=0 ft=1) fluid=1)


    Maximum depth: 0.131 (automatic)

    Speed of system: 2 (fixed)

    Speed of sound: max(coefsound*speedsystem,10*sqrt(g*depth))

    Speed of sound: 40 max(40,11.3363)

    Speed of sound: 40 (automatic)


    Coefh:1.000000 (hdp:1.732051)


    [PreparePoints-End time:19.158s]


    [SaveCase3D]

    CaseDambreak_out/CaseDambreak_MkCells.vtk (time:0.022s, shapes:187651, points:750604)

    CaseDambreak_out/CaseDambreak.bi4<double> (time:8.924s)

    CaseDambreak_out/CaseDambreak_All.vtk (time: 4.971s)

    CaseDambreak_out/CaseDambreak_Bound.vtk (time: 0.208s)

    CaseDambreak_out/CaseDambreak_Fluid.vtk (time: 5.672s)

    CaseDambreak_out/CaseDambreak.xml (time:0.026s)

     

    Data file copied: Floating_Materials.xml (xml row:108)

    Geometry file copied: soleira_1mm.stl (xml row:35)

    Geometry file copied: brecha3dist.stl (xml row:59)

    Geometry file copied: cunha-maior v1.stl (xml row:67)

    Geometry file copied: brecha3distblococai.stl (xml row:75)


    Save files time: 19.822s


    Particle limits:

     X range: -1.724 to 0.893 [m]

     Y range: 0 to 0.396 [m]

     Z range: -0.2 to 0.298 [m]

    [SaveCase3D-End time:26.356s]


    Finished execution (code=0).


    ----------------------------------------------------

    Run.out:


    DualSPHysics5 v5.0.175 (21-02-2021)

    ====================================

    Threads by host for parallel execution: 2

    [Initialising JSphCpuSingle 27-07-2021 19:57:39]

    ProgramFile=".../bin/linux/DualSPHysics5.0CPU_linux64"

    ExecutionDir=".../examples/main/CaseDamBreak_dp0.001_folga0.5mm"

    XmlFile=".../CaseDamBreak_dp0.001_folga0.5mm/CaseDambreak_out/CaseDambreak.xml"

    OutputDir=".../main/CaseDamBreak_dp0.001_folga0.5mm/CaseDambreak_out"

    OutputDataDir=".../CaseDamBreak_dp0.001_folga0.5mm/CaseDambreak_out/data"

    XML-App: GenCase v5.0.229 (29-01-2021)

    XML-Vars (uservars + ctes): CaseName=["CaseDambreak"] Data2D=[0] Data2DPosy=[0] H=[0.00173205] KernelSize=[0.0034641] B=[228571] Gamma=[7] Rhop0=[1000] Dp=[0.001] Gravity_x=[0] Gravity_y=[0] Gravity_z=[-9.81] MassFluid=[1e-06] MassBound=[1e-06]

    XML-Vars (parameters): TimeMax=[3] TimeOut=[0.01]

    Threads by host for parallel execution in Chrono: 1

    **Basic case configuration is loaded

    **Special case configuration is loaded

    Loading initial state of particles...

    Loaded particles: 86941999

    MapRealPos(border)=(-1.72409,-8.66025e-05,-0.200087)-(0.893087,0.396087,0.298087)

    MapRealPos(final)=(-1.72409,-8.66025e-05,-0.200087)-(0.893087,0.396087,0.547173)

    **Initial state of particles is loaded

    **3D-Simulation parameters:

    CaseName="CaseDambreak"

    RunName="CaseDambreak"

    Symmetry=False

    SavePosDouble=False

    SaveFtAce=False

    SvTimers=True

    Boundary="DBC"

    StepAlgorithm="Symplectic"

    Kernel="Wendland"

     Wendland.awen=8.040180e+07

     Wendland.bwen=-2.321011e+11

    Viscosity="Artificial"

     Visco=0.01

     ViscoBoundFactor=1

    DensityDiffusion="Fourtakas et al 2019 (inner)"

     DensityDiffusionValue=0.1

    Shifting="None"

    RigidAlgorithm="SPH+CHRONO"

    FloatingCount=1

    FtPause=0

    FtConstraints=False

    FtIgnoreRadius=False

    CaseNp=86941999

    CaseNbound=4134070

    CaseNfixed=4091727

    CaseNmoving=0

    CaseNfloat=42343

    CaseNfluid=82807929

    PeriodicActive="None"

    Dp=0.001

    KernelH=0.001732 (CoefficientH=1; H/Dp=1.73205)

    KernelSize=0.003464102

    CteB=228571.4

    Gamma=7

    RhopZero=1000

    Cs0=39.99999941406249

    CFLnumber=0.2

    DtIni=4.330127039558264e-05

    DtMin=2.165063552041092e-06

    DtAllParticles=False

    MassFluid=0.000001

    MassBound=0.000001

    FtPause=0

    TimeMax=3

    TimePart=0.01

    Gravity=(0,0,-9.81)

    NpMinimum=4134071

    RhopOut=True

    RhopOutMin=700

    RhopOutMax=1300

    WrnPartsOut=True


    *** Exception (JArraysCpuSize::AllocPointer) at JArraysCpu.cpp:86

    Text: Cannot allocate the requested memory.


    Finished execution (code=1).


    -----


    Regards.

  • AlexAlex

    The error is clear: : Cannot allocate the requested memory.

    You are trying to simulate 87 million particles...

    Regards

  • cadff1cadff1

    Yes, Alex.

    In the channel (with length = 1.8 m), I need to use a high number of particles. However, I am just using Dp = 0.001.

    And trying to improve the resolution to analyse the physical behaviour of the fluid flow.

    Do you have data relating to the number of particles and memory requirements?

    Also: is there any template for implementing an inlet in the channel?


    Regards.

    Carlos

  • cadff1cadff1

    Observation:

    As the flume has 1.8m, a high number of particles is used in the fluid discretization. There is no way to change this.


    Regards.

  • AlexAlex

    More info here:

    1) https://github.com/DualSPHysics/DualSPHysics/wiki/3.-SPH-formulation#315-open-boundary-conditions

    examples/inletoutlet


    2) https://twitter.com/DualSPHysics/status/1421896245308764168?s=19

  • cadff1cadff1

    Firstly, thank you for your willingness to help, Alex.

    ....

    Good afternoon!

    I need information that only software developers can provide.

    I explain: I'm not having success running the simulations on the server. Enough memory is not being allocated, in simulations using CPU. When requesting more memory in the submission (20GB, for example) and a specific number of cores (10, for example), a single core and 2GB are being used. I was guided to examine the code to change the parameters that define the parallel processing in DualSPHysics.

    In this context, I ask for your help in defining the parameters that allow the use of more cores (and memory) in the parallel processing of the simulation.

    Regarding the use of the GPU on the server, information that you can make available is also welcome.

    Thank you for your attention, again.

    With best regards.

Sign In or Register to comment.

© Vanilla Keystone Theme 2024