Skip to content
Popular Search tm5modbusvisionprofinettm-

How to Use AI Instance Segmentation for 3D RBP – Take De-Palletizing Application As an Example

Table of Contents

Examples are valid for :

TMflow Software version : 2.18 or above

TM Robot Hardware version : HW3.2 or above

Other specific requirements :

Note that older or latest software versions may have different results

Establish Environment #

  • Collecting TM carton, stacking carton on the stand, and collecting image from 3D camera
  • Cartons with the same apearance are palced for testing, with the TM logo facing up (as shown below)

Collecting Image Through the TMflow #

  • Open TMflow, select configuration –> AI+

  • Enter account, password and connect IP for the AI+ Training Server you are using, select AI+ Training Server, and press “Save”

  • Open your de-palletizing project

  • Find Positioning vision job in the de-palletizing project

  • Edit the vision job

  • Set initiate parameter for the 3D camera and enable image color to view 2D image, and press “Save” 

 

  • Set the ROI to ensure that your carton are within the detection range

  • Choose Geometry method in Find module

  • Press Setting to choose or create the dataset where your image will be stored on AI+ Training Server

  • Confirm that the arrangement of the cartons matches the desired image, press “Save Image”, and this concludes the collection in TMflow

  • You can check within the AI+ Training Server to see if your images have been successfully collected in your datasets

Recommendations for Collecting Images of Cartons #

  • There are 12 essential scenarios to collect, explained as follows:
    • Three Rotation Scenarios (Total 8 images):
      • Whether rotate the carton by 45 degrees
      • Whether increase the height(raise to the maximum possible height)
      • Whether apply adhesive to the carton(In the adhesive scenarios, please include all possible cartons for the first layer placement)
    • Stacking Scenarios(Total 4 images):
      • Two cartons at the bottom, one on the top. Whether rotate the top carton by 45 degrees
      • Four cartons at the bottom, one on the top. Whether rotate the top carton by 45 degrees
  • In addition to the specified 12 images, it is recommended to randomly collect various scenarios until reaching a total 20 images or above to ensure stability in assessment
  • After deploying the model, there are still scenarios that do not meet the requirements, you can collect images of these scenarios and retrain the model

Image Collection Example #

  • No rotate, no increase the height, no adhesive

  • Rotate 45 degrees, no increase the height, no adhesive

  • No rotate, increase the light, no adhesive

 

  • No rotate, no increase the height, adhesive cartons

  • Rotate 45 degrees, increase the height, no adhesive

  • Rotate 45 degrees, no increase the height, adhesive cartons

  • No rotate, increase the height, adhesive cartons

  • Rotate 45 degrees, increase the height, adhesive cartons

  • Stacking scenario, two cartons at the bottom, no rotate

  • Stacking scenario, two cartons at the bottom, rotate 45 degrees

  • Stacking scenario, four cartons at the bottom, no rotate

  • Stacking scenario, four cartons at the bottom, rotate 45 degrees

  • In addition to the mentioned 12 images, it is recommended to collect 20 or more images similar to the following scenarios to ensure the stability of the model

Label the Images and Train the Model #

  • Enter the account and password to log into the AI+ Training Server

  • Establish your dataset on the AI+ Training Server and collect images from TMflow to this dataset, or alternatively, create a dataset directly on TMflow and collect images.

 

  • Choose your dataset and create the label to the cartons

  • Click on the “Annotate” and begin annotating your cartons

  • Add new instance and use the Annotate tool to label your cartons

  • Please annotate cases where the entire top surface of the carton is visible, and it is advisable not to annotate cases where the corners are covered or incomplete
  • Create the project to train the model

  • Choose your dataset

  • Next, configure the model parameters, you can refer to the following parameters for settings to ensure that various scenarios are taken into account

  • After the model training is complete, you can download the model to your hard drive and then load the model into TMflow. Alternatively, you can directly import the model in TMflow

Import AI Model into TMflow #

  • Open TMflow, select configuration –> AI+

  • Enter your account and password, connect to the AI+ Training Server, choose AI+ Instance Segmentation(3D), then select the model and click download

  • Open your de-palletizing project

  • Find positioning vision job in the de-palletizing project

  • Edit the vision job

 

  • Set initiate parameter for the 3D camera and enable image color to view 2D image, and press “Save”

  • Set the ROI  to ensure that your carton are within the detection range

  • Choose Geometry method in Find module

  • Press Setting to choose or create the dataset where your image will be stored on AI+ Training Server

  • Choose your AI model

  • Ensure the model can successfully locate the cartons, finish the vision job and save the results
  • After completing the vision job, proceed to execute the de-palletizing project to test the success of the task

 

 

 

 

 

What are your Feelings

Powered by BetterDocs