The virtual bag generator block in the DEBISim pipeline is responsible for creating the 3D virtual bags for which the CT projection data and image reconstructions are generated by the rest of the pipeline. This is a powerful feature of DEBISim as virtual bags can be created using this block either (i) manually by means of a GUI console, (ii) for standard phantoms with user-defined shape lists or (iii) in a randomized mode to create baggage datasets. This page describes the Shape List data format used for annotating virtual bags and then describes the various submodules used by this block to create the virtual bags.
The shape list is a data format used in DEBISim to store the data for the different objects spawned within a virtual bag. A Shape List is basically a list of Python dictionaries each describing a single object within the image - the SL dictionary for each object contains (i) the geometric specifications of the objects, (ii) the positions and orientation of the object and (iii) material properties of the object. The detailed description of a shape list is given below:
Each object dictionary in an SL contains the following key-value pairs:
‘label’ - integer label of the object within the reference image.
‘shape’ - shape of the object - current version includes the following options:
| ‘E’ | Ellipsoid |
| ‘Y’ | Cylinder |
| ‘C’ | Truncated Cone |
| ‘B’ | Box |
| ‘S’ | Deformable Sheet |
| ‘M’ | A custom shape (specified by a 3D binary mask - used to include special shapes or 3D data read from .fits.gz files) |
| ‘T’ | Fixed objects within the scanner setup (table, tray and baggage boundary) |
‘geom’ - dictionary containing geometric specifications for the object shape - varies for each object shape:
| ‘E’ | ‘center’ | Center co-ordinates [x,y,z] of the ellipsoid |
| ‘axes’ | principal axes [a1, a2, a3] | |
| ‘rot’ | rotation (Euler) angles in degrees [t1, t2, t3] | |
| ‘Y’ | ‘base’ | Center co-ordinates of the cylinder base |
| ‘apex’ | Center co-ordinates of the cylinder apex | |
| ‘radius’ | radius of the cylinder | |
| ‘C’ | ‘base’ | Center co-ordinates of the cone base |
| ‘apex’ | Center co-ordinates of the cone apex | |
| ‘radius1’ | starting radius of the cone | |
| ‘radius2’ | ending radius of the cone | |
| ‘B’ | ‘center’ | Center co-ordinates [x,y,z] of the box |
| ‘dim’ | dimensions of the box [a1, a2, a3] | |
| ‘rot’ | rotation (Euler) angles in degrees [t1, t2, t3] | |
| ‘S’ | ‘center’ | Center co-ordinates [x,y,z] of the sheet |
| ‘dim’ | dimensions of the undeformed sheet [a1, a2, a3] | |
| ‘rot’ | rotation (Euler) angles in degrees [t1, t2, t3] | |
| ‘M’ | ‘center’ | Center co-ordinates [x,y,z] of the shape |
| ‘dim’ | bounding box dimensions of the shape [a1, a2, a3] | |
| ‘rot’ | rotation (Euler) angles in degrees [t1, t2, t3] | |
| ‘mask’ | 3d object mask | |
| ‘src’ | source .fits.gz file |
‘material’ - material assigned to the object - can be selected from the existing materials from the database in the ./include/mu/ directory or can be created using the MuDatabaseHandler class.
‘mu_dict’
- dictionary containing the X-ray coefficient values of the assigned material. The dictionary is directly copied from the MuDatabaseHandler and has the following key-value pairs: {'compton':, 'pe':, 'z':, 'lac_1': , 'lac_2':, 'HU_1':, 'HU_2': } . See MuDatabaseHandler for details.
‘mu_curve’ - Attenuation Curve for the given object material.
‘lqd_flag’ - Set to True if the object is a liquid-filled container.
‘lqd_param’ - Parameters for the liquid filled container - specified only if lqd_flag is set to True:
| ‘cntr_thickness’ | thickness of the liquid container |
| ‘lqd_level’ | level to which the liquid is filled (specified as a fraction from 0 to 1) |
| ‘lqd_label’ | liquid label |
| ‘lqd_material’ | liquid material |
| ‘mu_curve’ | Attenuation Curve of the liquid material. |
| ‘mu_dict’ | Same as 'mu_dict' except this contains the coefficients for the liquid material. |
The Shape List basically provides the information on the objects’ geometry and material composition which can then be used to annotate the image reconstructions generated at the end of the DEBISim pipeline. The simulator in both the randomized mode or the user-interactive mode creates and saves a Shape List for any virtual bag it creates. The label image for a virtual bag can in fact be constructed entirely (with some exceptions) from the shape list by using the ImageVoxelizer3D module - this is useful for simulating well-known phantoms whose geometric structure can be derived from primitive shapes.
The shape list can be created in one of the three ways:
The module ShapeListHandle contains support for creating, handling and managing Shape Lists used in DEBISim simulations. It can be imported and used to produce user-specified Shape Lists.
from ShapeListHandle import *
from numpy import *
slh = ShapeListHandler()
# Creates a sample shape list for an acrylic ellipsoid pierced with two
# metal rods
sample_shape_list = [
slh.create_sim_object('B',
[
('center', array([100, 100, 100 ])),
('dim', array([10, 10, 80])),
('rot', array([0, 90, 0]))
],
'Al'),
slh.create_sim_object('B',
[
('center', array([100, 100, 100 ])),
('dim', array([10, 10, 80])),
('rot', array([90, 0, 0]))
],
'Cu'),
slh.create_sim_object('B',
[
('center', array([100, 100, 100 ])),
('dim', array([50, 50, 50])),
('rot', array([0, 0, 45]))
],
'acrylic')
]
To know more using ShapeListHandler, refer to the Python API Reference: ShapeListHandler
Randomized 3D baggage generation in DEBISim can de done using the BaggageCreator3D module for automated generation of large baggage datasets. The module BaggageCreator3D uses two classes for creating randomized bags: (i) The class Object3D() is used for generating 3D object shapes that are to be projected onto the virtual bag while (ii) the class BaggageImage3D() arranges the projected objects within the baggage volume to prpduce the 3D virtual baggage image. Baggage generation is thus done by creating a list of Object3D() shapes from a Shape List and then feeding them to BaggageImage3D() to be arranged in the virtual bag. The code for arranging the objects makes use of rigid body constraints and rules in order to yield a realistic baggage simulation. The constraints and rules used in the code mimic the physical interlay between different objects placed in a real bag and are enumerated as follows:
Overlap rule:
It ensures that no two objects overlap each other. Whenever a new object
is found to be overlapping with another object, it is translated by an
amount equivalent to the superimposed volume so the overlap is
eliminated.
Gravity rule:
It adds a downward translation component to each object till it hits the bottom of the bag or another object at rest.
Liquid container Rule:
This creates an liquid filled container within the bag. In order to do
this, the object SL dictionary must contain the required liquid
parameters, namely, the liquid level, the lquid material and the
container thickness.
Sheet Rule:
It allows generation and placement of deformable sheet type objects in
the baggage image. When spawned in the bag, the sheet mesh is stretched
and deformed till it settles down within the bag without penetrating
another object.
BaggageImage3D() arranges the Object3D() shapes within the bag by recursively applying these rules in the following priority :
This continues till all the objects settle down in the virtual bag - the positions and orientations of the object are continuously updated as the objects are rearranged within the bag. The result is a virtual bag consisting of a 3D label image of the baggage volume and an updated Shape list containing the final positions and orientations of the objects within the bag.
Both the classes Object3D and BaggageImage3D are explained in the next section with examples of how to simulate randomized 3D virtual bags.
As mentioned before, the class Object3D() used for generating 3D object shapes that are to be projected into the virtual bag. An Object3D() instance is initialized by reading in an object dictionary from a Shape List (See the section on Shape Lists). The different object shapes that can be created using this class are:
The following example shows how the primitive shape objects are generated using this class:
To know more using Object3D, refer to the Python API Reference: Object3D
The class BaggageImage3D() is responsible for the placement of different objects within the baggage volume to produce the final 3D virtual bag. As mentioned in the previous section, Baggage generation is done by creating a list of Object3D() shapes from a Shape List and then feeding them to BaggageImage3D() to be arranged in the virtual bag using rigid body constraints and rules in order to yield a realistic baggage simulation. The four rules that are followed while arranging the objects, namely,
from ShapeListHandle import *
from BaggageCreateor3D import *
sl_handle = ShapeListHandle()
# Creating an ellipsoid made of polyethylene
obj_dict = sl_handle.create_sim_object(
[ ('center', array([0, 0, 0])),
('axes', array([30, 100, 30])),
('rot', array([20, 50, 30]))
],
'E',
'polyethylene', 2)
curr_obj = Object3D(obj_dict, array[(100,100,100)])
# Creating an box made of Zinc
obj_dict = sl_handle.create_sim_object(
[ ('center', array([0, 0, 0])),
('dim', array([30, 100, 30])),
('rot', array([20, 50, 30]))
],
'B',
'Zn', 4)
curr_obj = Object3D(obj_dict)
# Creating an truncated cone made of Aluminum and filled with water
lqd_params = dict(lqd_material='water', lqd_label=5,
lqd_level=0.5, cntr_thickness=3)
obj_dict = sl_handle.create_sim_object('C',
[ ('base', array([10, 15, 10])),
('apex', array([70, 90, 80])),
('radius1', 30),
('radius2', 60)
],
'C',
'polyethylene', 6,
lqd_flag=True,
lqd_param=lqd_params)
curr_obj = Object3D(obj_dict, array([0,0,0]))
are observed in the order of decreasing priority. Hence the liquid-container / sheet rule are checked for first while overlap and gravity are checked for last. Because overlap rule has the lowest priority, baggage images with very high clutter (>30 objects) can sometimes produce a case or two of overlapping object shapes.
Generally, baggage generation using this class can be run by calling the method BaggageImage3D.create_baggage_image() while feeding in a list of Object3D instances (this list is created by reading the object dictionaries from a Shape list one-by-one). To facilitate randomized generation of baggage images with similar objects and materials, the method BaggageImage3D.create_random_object_list() can be used which automatically generates a random shape list and places the objects in the virtual bag.
The class is used to create a randomized virtual bag - the script can be run iteratively to create a randomized dataset with the same material specifications as follows:
from BaggageCreator3D import *
from Shape ListHandle import *
import numpy as np
from util import *
prior_image = np.zeros((350,350,350))
prior_image[250+120:250+165, 250+120:250+165, 100:400] = 50
vol_bag = BaggageImage3D(prior_image=prior_image)
material_list = ['Al', 'pyrex', 'acetal', 'C', 'polyethylene',
'Cu', 'acrylic', 'neoprene', 'Mg']
liquid_list = ['water', 'HCl', 'alcohol']
material_pdf = [0.1, 0.2, 0.1, 0.05,0.05,0.23,0.1,0.17]
liquid_pdf = [0.3, 0.5, 0.3]
obj_list = vol_bag.create_random_object_list(material_list,
liquid_list,
material_pdf,
liquid_pdf,
lqd_prob=0.2,
sheet_prob=0.5,
min_dim=10,
max_dim=90,
number_of_objects=20,
spawn_liquids=True,
spawn_sheets=True)
vol_bag.create_baggage_image(obj_list)
slide_show(vol_bag.full_bag_vol, vmin=0, dt=0.0001)
To know more using BaggageImage3D, refer to the Python API Reference: BaggageImage3D
This class is used to facilitate generating a 3D virtual bag by directly reading the shape list. Unlike BaggageImage3D which generates the shape list after arranging all the objects within the bag, ImageVoxelizer3D creates a 3D label image by just placing the objects from the Shape list. The class is useful for creating simulations from previously shape lists or for pre-defined shape lists of standard phantom (You can find theses in the ./examples/phantom_shape_lists/ directory in the software.
The class ImageVoxelizer3D is initialized with an input shape list and the 3D image dimensions of the bag. Calling the method self.voxelize_3d_image() then generates a 3D label image wherein each label corresponds to the shape list label of the object.
from ImageVoxelizer3D import *
from MuDatabaseHandler import *
from util import *
import pickle, torch
file_name = './examples/phantom_shape_lists/battelle_phantom_a.pyc'
with open(file_name, 'r') as f:
data = pickle.load(f)
f.close()
Mu = MuDatabaseHandler()
shape_list = [data[int(x)] for x in shape_list.keys()]
voxelizer = ImageVoxelizer3D(shape_list, imgshape=(664,664,886),
mu_dbase=Mu)
# The ground truth label image and corresponding compton image are created as the output
gt_image, compton_image = voxelizer.voxelize_3d_image()
slide_show(gt_image.cpu().numpy())
Note:
The simulator saves the shape list as a pickle .pyc file - when read using the pickle.load(), the file is loaded as a
dictionary with the object labels as the keys. The read data can be converted to a shape list using the following code:
file_name = './include/examples/phantom_shape_lists/acr_phantom.pyc'
with open(file_name, 'r') as f:
data = pickle.load(f)
f.close()
shape_list = [data[int(x)] for x in shape_list.keys()]
To know more using ImageVoxelizer3D, refer to the Python API Reference: ImageVoxelizer3D