Acquiring image data from ROS

ROS images are available in various different formats, and depending on the camera in question, these images are useful for different kinds of sensing. This tutorial describes how to acquire camera data from ROS and utilize it in Simulink, for the purposes of utilizing the code generator and other toolbox options within Simulink. You can find a video that runs this tutorial at https://youtu.be/_QPucsSbvAU

Get the tutorial model

Download imageAcquisition.slx from the catvehicle_simulink repository, or clone the entire repository:

git clone https://github.com/sprinkjm/catvehicle_simulink.git

This tutorial model shows how to acquire a raw ROS image into Simulink, and then transform that image into a format that Simulink can use. 

How a ROS image becomes a MATLAB one

A raw ROS image is made up of 3x value for each pixel, starting in the upper-left-hand corner and proceeding to the end of each line. The result is a single vector of length width*height*3. In MATLAB/Simulink, the format of the image is a matrix in R^3 with dimensions [width, height, 3]. In order to enable this transformation, we need a little custom code on the "output" side of the bus routing block. Look inside the subsystem that produces the image to see a MATLAB function block, with the below contents.

function [R,G,B] = fcn(u)
%#codegen
Rs = u(1:3:end);
Gs = u(2:3:end);
Bs = u(3:3:end);
R = reshape(Rs,800,800)';
G = reshape(Gs,800,800)';
B = reshape(Bs,800,800)';

This produces an 800x800 "image" or red, green, and blue colors. The output vector concatenate puts these into an 800x2400 matrix, which we then reshape into an 800x800x3 matrix. The result "Image" is a color value that Simulink can display.

Look at the images in Simulink

Take an example bagfile you have, or use one from the Files section of the website. Start up roscore.

roscore

Play the bagfile,

rosbag play bagfile.bag

then push "Play" in Simulink. You may need to click on the "To Video Display" in order to see it while Simulink is executing.

How to take advantage of various toolboxes for computer vision?

You'll probably want to convert the image to grayscale, and then start using whatever image processing or computer vision algorithms you'd like. At this point, you're ready to start innovating with those toolboxes, knowing that you can acquire images from either your simulator, or a bagfile with simulated/actual data.