First time here? Checkout the FAQ!
0 votes
by (180 points)

Hello everyone, I am very new to openCARP and I'm trying to run my first mini project for bidomain. I have a tetrahedral mesh of a sphere (tag=0) inside a block (tag=1), which is supposed to represent an atrium in a torso respectively. I want to run Courtemanche model starting somewhere within the sphere and propagate everything onto the block. I used another software to generate surface meshes, but later I used meshtool to combine them into tetrahedral and to generate fibres. I confirmed tags with meshtool query tags:

Reading mesh: combined_tet

Myocardium tags:


Bath tags: 


Myocardium tags: 

0: 14026441 elems

Bath tags: 

1: 346939 elems

Later, I generated the following .par file based on the examples and openCARP Manual - Tissue Simulation: Running in plain mode (not sure if my .par file looks correct, so I would appreciate if someone could also confirm whether it looks legit):

meshname = ./meshes/combined_tet   

# ----------------------------------------------------------------

num_phys_regions = 2 

phys_region[0].name = "Extracellular"

phys_region[0].ptype = 1

phys_region[0].num_IDs = 2 

phys_region[0].ID[0] = 0 # atrium

phys_region[0].ID[1] = 1 # torso 

phys_region[1].name = "Intracellular"

phys_region[1].ptype = 0 

phys_region[1].num_IDs = 1

phys_region[1].ID[0] = 0

num_imp_regions = 1

imp_region[0].name = "Atrium"

imp_region[0].im = Courtemanche

imp_region[0].im_param = "GCaL-55%,GK1+100%,factorGKur-50%,Gto-65%,GKs+100%,maxIpCa+50%,maxINaCa+60%,GKr*1.6"

imp_region[0].num_IDs = 1

imp_region[0].ID = 0

num_gregions = 2

gregion[0].name = "Atrium"

gregion[0].g_il = 2.9135

gregion[0].g_it = 0.3526

gregion[0].g_in = 0.118

gregion[0].g_el = 1.0283

gregion[0].g_et = 0.4702

gregion[0].g_en = 0.4262

gregion[0].num_IDs = 1

gregion[0].ID = 0

gregion[1].name = "Torso"

gregion[1].g_bath = 0.22

gregion[1].num_IDs = 1

gregion[1].ID = 1

# ----------------------------------------------------------------

num_stim = 1

stimulus[0].stimtype = 0

stimulus[0].strength = 30

stimulus[0].duration = 2.0

stimulus[0].start = 0

stimulus[0].x0 = -15640.7

stimulus[0].xd = 2000.0

stimulus[0].y0 = 6291.57

stimulus[0].yd = 2000.0

stimulus[0].z0 = 21792.4

stimulus[0].zd = 2000.0

# ----------------------------------------------------------------


tend = 500

spacedt = 1.0

timedt = 1.0

gridout_i = 3

gridout_e = 3

bidomain = 1

dt = 20

Finally, I am running openCARP +F default.par and I am getting the following error (I found a similar question L3 error tag # not found in element list but the answer was not really helpful):

L1 : Warning: Legacy stimuli defined. Please consider switching to stimulus definition "stim[]"!

    *** Processing meshes ***

Reading reference mesh: ./meshes/combined_tet.*

Done in 8.020109 sec.

Done in 0.032718 sec.

Processing submeshes

All done in 18.557148 sec.

Writing "Intracellular" surface:

Writing "Intracellular" mesh: combined_tet_i

Writing "Extracellular" surface:

Writing "Extracellular" mesh: combined_tet_e

    *** Initializing physics ***

Initializing Electrics ..

Setting up ionic models and plugins


Assigning IMPS to tagged regions:

Ionic model: Courtemanche to tag region(s) [0].

Ionic model: Courtemanche

GCaL                 modifier: -55%            value: 0.05571

GK1                  modifier: +100%           value: 0.18

factorGKur           modifier: -50%            value: 0.5

Gto                  modifier: -65%            value: 0.05782

GKs                  modifier: +100%           value: 0.258

maxIpCa              modifier: +50%            value: 0.4125

maxINaCa             modifier: +60%            value: 2560

GKr                  modifier: *1.6            value: 0.04704

L3 : Region tag 1 in gregion_i[1] not found in element list for Intracellular grid.

L4 : Missing element tags in element file!

Check region ID specs in input files!

application called MPI_Abort(compute, 59) - process 0

[unset]: write_line error; fd=-1 buf=:cmd=abort exitcode=59


system msg for write_line failure : Bad file descriptor

I checked .elem files as well and I can see both tag 1 and tag 0 in there, so I am running out of the ideas. Maybe I did something wrong in .par file?

Thank you I advance!

1 Answer

0 votes
by (7.8k points)

There is no obvious error in your parameters. You might want to check the petsc_error.log file in your simulation output directory. Maybe there is some further information on the MPI abort. The warnings regarding missing regions are not the reason for the abort.

For anyone to be able to help you further, you need to make the experiment available.

Best, Aurel
by (180 points)
Hi Aurel,

I checked petsc_error.log file and it says:

[0]PETSC ERROR: --------------------- Error Message --------------------------------------------------------------
[0]PETSC ERROR: MatSeqAIJSetPreallocation_SeqAIJ
[0]PETSC ERROR: See for trouble shooting.
[0]PETSC ERROR: Petsc Release Version 3.17.1, Apr 28, 2022
[0]PETSC ERROR: openCARP on a  named bioeng563-pc by jg23 Tue Jan  2 12:49:38 2024
[0]PETSC ERROR: Configure options PETSC_ARCH=docker-opt --prefix=/usr/local/lib/opencarp/lib/petsc --download-mpich --download-fblaslapack --download-metis --download-parmetis --download-hypre --with-debugging=0 COPTFLAGS=-O2 CXXOPTFLAGS=-O2 FOPTFLAGS=-O2
[0]PETSC ERROR: #1 PetscMallocAlign() at /tmp/petsc-3.17.1/src/sys/memory/mal.c:49
[0]PETSC ERROR: #2 PetscMallocA() at /tmp/petsc-3.17.1/src/sys/memory/mal.c:401
[0]PETSC ERROR: #3 MatSeqAIJSetPreallocation_SeqAIJ() at /tmp/petsc-3.17.1/src/mat/impls/aij/seq/aij.c:4032
[0]PETSC ERROR: #4 MatSeqAIJSetPreallocation() at /tmp/petsc-3.17.1/src/mat/impls/aij/seq/aij.c:3964
[0]PETSC ERROR: #5 MatCreateAIJ() at /tmp/petsc-3.17.1/src/mat/impls/aij/mpi/mpiaij.c:4381
[0]PETSC ERROR: ------------------------------------------------------------------------
[0]PETSC ERROR: Caught signal number 11 SEGV: Segmentation Violation, probably memory access out of range
[0]PETSC ERROR: Try option -start_in_debugger or -on_error_attach_debugger
[0]PETSC ERROR: or see
[0]PETSC ERROR: or try on GNU/linux and Apple MacOS to find memory corruption errors
[0]PETSC ERROR: configure using --with-debugging=yes, recompile, link, and run
[0]PETSC ERROR: to get more information on the crash.
[0]PETSC ERROR: #6 User provided function() at unknown file:0
[0]PETSC ERROR: Run with -malloc_debug to check if memory corruption is causing the crash.

Have you seen this error before? I tried to google this MPI errors and it seems that it s quite software specific, because different softwares have different solutions. Is there any way I could run openCARP in the debug mode?

My system is Ubuntu 22.04 on AMD Ryzen 9 with Debian Open MPI 4.1.2 and MPI API 3.1.0
by (180 points)
Additionally I tried to run the simulation with carputils to get more details on used PETSC and MPI (just to check if it is linking with correct files and paths, because I also have installed Paraview). It gives me the same error, despite the paths pointing to things installed within openCARP:

   CARP:              /usr/local/bin/openCARP
   MESHER:            /usr/local/bin/mesher
   BENCH:             /usr/local/bin/bench
   IGBHEAD:           /usr/local/bin/igbhead
   IGBOPS:            /usr/local/bin/igbops
   IGBEXTRACT:        /usr/local/bin/igbextract
   IGBAPD:            /usr/local/bin/igbapd
   IGBDFT:            /usr/local/bin/igbdft
   SV2H5B:            /usr/local/lib/python3.10/dist-packages/bin/
   SV2H5T:            /usr/local/lib/python3.10/dist-packages/bin/
   CLOSEST_HC:        /usr/local/lib/python3.10/dist-packages/bin/closest_hc
   APDRESTITUTION:    /usr/local/lib/python3.10/dist-packages/bin/APDrestitution
   TUNECV:            /usr/local/lib/python3.10/dist-packages/bin/tuneCV
   RESTITUTECV:       /usr/local/lib/python3.10/dist-packages/bin/restituteCV
   GENERATEMETADATA:  /usr/local/lib/python3.10/dist-packages/bin/generateMetadata
   MESHTOOL:          /usr/local/bin/meshtool
   LIMPETFE:          /usr/local/bin/
   MAKEDYNAMICMODEL:  /usr/local/bin/
Command Line Options
   NP:        1
   BUILD:     CPU
   FLAVOR:    petsc
   DRY:       False
Solver Option Files
   Parabolic: /usr/local/lib/python3.10/dist-packages/carputils/resources/petsc_options/ilu_cg_opts
   Elliptic:  /usr/local/lib/python3.10/dist-packages/carputils/resources/petsc_options/gamg_cg_opts
   OPENCARP_SRC:  /usr/local/lib
   MPIEXEC:       /usr/local/lib/opencarp/lib/petsc/bin/mpiexec
System Info
   Python:    3.10.12
   OS:        Ubuntu 22.04.3 LTS
   SETTINGS:  /home/jg23/.config/carputils/settings.yaml
   Name:      Your Name
   ORCID:     0000-0000-0000-0000

mkdir -p /home/jg23/Desktop/test-carputils/2024-01-02_petsc_np1
by (7.8k points)
It looks like the matrix allocation failed. Are you sure that you have enough memory for the simulation?
by (180 points)
I improved  the mesh quality and I made it less corse and it run. I am wondering whether the previous mesh was so bad that the solver couldn't do computations (after using meshtool quality check the histogram had a little bump in 0.9-1.0 region, now everything is within 0.0-0.4)
by (7.8k points)
Maybe the initial mesh had topological issues. You could run "meshtool query edges" and look at the number of connections. Maybe you see something suspicious in the Min,Max,Mean values (e.g. 0/NaN/Inf). Best, Aurel
Welcome to openCARP Q&A. Ask questions and receive answers from other members of the community. For best support, please use appropriate TAGS!
architecture, carputils, documentation, experiments, installation-containers-packages, limpet, slimfem, website, governance