A “journalist robot” that evaluates newsworthiness and writes articles

Associate Professor Tatsuya Harada
(Department of Mechano-Informatics)

Associate Professor Harada is a robot researcher focusing on the development of a “journalist robot” that performs journalists’ tasks by giving it news coverage know-how (intelligence).  Although the final goal is a robot that detects newsworthiness from events in its surroundings and writes articles, it is currently in the phase where it acquired abilities to instantaneously describe images it sees in words using image annotation technology.  This robot is a milestone towards his aspiration for the intelligent next-generation robot.

The journalist robot is considered a bridge to connect the real and cyber worlds and expected to quickly evaluate whether it is “newsworthy” or “irrelevant” from images taken during interviews in the future.  For example, it detects anomaly from a flower placed on the table in the corridor and describes “corridor,” “window,” “flower,” “table,” and “floor” captured with its camera in words as it nears them.  This is where image annotation technology plays an important role.

In order for the robot to check the flower on the table by picking it up with its hands, another core technology called segmentation technology is required to clip the flower from the image.  Since the robot cannot grab an object without knowing its shape, information such as its position and shape needs to be accurately communicated.  It is important to note that in Associate Professor Harada’s image segmentation research “there is no model in the robot to determine the object shape.”  This enables the robot to develop intelligence to handle various forms of objects including crushed PET bottles and soft clothes.

The journalist robot is “information support robot” as it describes what it sees in words with image annotation technology.  If it is combined with image segmentation technology which provides “physical support,” it may become a reality to see the “next-generation robot” that can bring an object such as a mobile phone as you request while having a conversation about interesting daily events.