Implementing Humanoid Nao Robot Therapies’ Children with Down Syndrome in a Smart City

Introduction

Down’s Syndrome

What is the Down’s Syndrome? : Today the number of related genetic abnormalities is quite high; to mention that there are more than 12,000 described genetic syndromes, on of them is Down Syndrome. In 1866 it was John Langdon Down who described for the first this syndrome, and on that year, it was attributed to a delay in the normal development. Subsequently it was stated that Down’s Syndrome was a consequence of infectious processes, alcoholism, between others. It was until 1958 when Jerome Lejeune and Pat Jacobs, discovered the presence of a third chromosome in the 21 pair of all cells, since then it’s considered a genetic syndrome. Being the first genetic abnormality described on human beings [1]. Down’s Syndrome creates deficiencies in physical and intellectual development among individuals. Children with Down Syndrome usually show physical characteristics, neuropsychological, sensorial, motor and cognitive quite similar. The last are described by Fernandez Sampedro and others (1993):

a. Attention is unstable, scattered and fatigable; to make a progress in the child’s learning, it must involve different activities to keep it.

b. Short term memory lacks of sensorial & hearing information processing which usually helps to improve with visual efforts. Regarding long term memory, there’s a challenge to store and recover information. The child retains memory by observation because of habit, but lacks of memory, which hardens language and vocabulary learning.

c. There’s a considerable retardation language wise in comparison to other development fields. There’s a gap between comprehension and expression levels.

d. Vocabulary delay is present, possibly because there’s a lack of comprehension in the relation between objects, people, facts, and words that represent them, there’s no retention of this relationship or there’s not a space-time frame acknowledgement.

e. Slowness in reaction timing. Children with Down’s Syndrome also show impulsivity, low tolerance to frustration, lack of persistence on tasks, low motivation innate and the need of external supervision to finish a task.

Although all the deficits are marked, some authors state that, when therapy programs integrate motivational aspects, there’s an improvement of their intellectual execution. Therefore, it is stated that they posses some learning skills, but, in comparison to their No-Down counterparts, they show instability in knowledge attainment, and this last one is slower.

What Problems Kids Face for an Education? : ”Down’s Syndrome is the most frequent cause of psychic disability, the estimated relation in cases is one in very 1000 - 1100 births, according to the World’s Health Organization. In Mexico, the Health Department, through the Gender Equality Center and Reproductive Health, in their Technical Guidelines for the Integral Attention for people with Down Syndrome, estimates that there’s a case in every 650 first borns. In Mexico there are many institutions that support children with Down Syndrome, generally there are specialized institutes for motor development and others in intellectual development [2]. However, most of the public schools since kindergarten won’t take in kids with different intellectual skills due to lack of prepared personnel to provide proper education to kids with DS, which leads to, mostly, to kids with DS to not receive education. In Mexico only 3% of kids with DS receive education. The main reason is the number of educators and equipment in specialized institutions are not enough to cover the number of kids with DS and as a secondary reason, Mexico is a country with low resources that’s why it’s complicated to take a kid to a specialized institution, and there might be the case that this institution is a few hours away from home.

What is NAO?: Throughout the last years, there has had a great growth on the interest in robots which are able to complete the task of a human assistance. They have been developed to be an aid in homes, for sanitary and therapeutic assistance. The use of the Nao humanoid robot is proposed to be implemented in the education for kids with DS, altogether with parents and professors who may teach them daily. The robot has two cameras, four microphones, nine tactile sensors, two ultrasonic sensors, eight pressure sensors to perceive the environment where it interacts. It has twenty-five grades of liberty which allow the robot has greater movement range making it ideal for any environment. Also, it has a voice synthesizer and two speakers that permit the communication between the robot and the user as it is described on (Figure 1).

Figure 1: Nao Sensors and Joints.

What Can NAO Do?: The main objective of this article is to develop certain cognitive skills through the use of the Nao humanoid robot, essentially color recognition and motor coordination. Besides, it will try to motivate the Down infant to speak and collaborate, since these are common social abilities that every infant should learn during his/her development. It has been achieved by the illustration process to recognize all types of colors through the Nao camera, which will mention the color of toys every time the user shows a figure to the humanoid. Moreover, it will be placed each piece inside a container at the beginning or at the end of a therapy session in hand with the humanoid robot to storage the used objects as part of a game.

Applications of the NAO to Help People with Down’s Syndrome

Color Recognition: In order to make the recognition, the humanoid was placed in front of a white table which measures 15x41x25 cm (WxLxH). As it has been said, the Nao humanoid owns two HD cameras, in this case, the making decision was to use the superior camera due to the position of the robot at the beginning of the action. But any of both may be taken. Regarding the objects, four figures of color red, blue, purple and orange were utilized. For the illustration process, Numpy module was used since it allows matrix working in Python and OpenCV, a computerized visionary library. Those were imported to Python 2.7 Software. The first step was to make the connection of the Nao camera through the computer using the following code:

Dispositivo_Video = ALProxy (‘ALVideoDevice’, “IP”, 9559)

AL_kTopCamera = 0

AAL_kQVGA = 1

AL_kBGRColorSpace = 13

captureDevice = Dispositivo_Video. subscribeCamera(“Prueba”, AL_kTopCamera, AL_kQVGA,

AL_kBGRColorSpace, 10)

Ancho = 320

Altura = 240

Imagen = np. zeros ((Altura, Ancho, 3), np. uint8)

The above code specifies the use of the superior camera, space of RGB color and a resolution of 320x240. Each pixel is saved on a new matrix of the same resolution. Once the matrix is obtained, the space of BGR color has to be change to HSV color. The model HSV is based on cylindrical coordinates and it is derived from RGB model. In addition, it represents colors combining three values which are: hue, saturation and value. The first one let color distinguishes each other, taking in account the length of the wave that goes from 0º to 360º. Nevertheless, the saturation refers to a sensation that goes from high to low color intensity. To finalize, the value refers to the amount of white that a color may contain. Previous description is shown in Figure 2 below. The next step has the purpose to construct color masks. In this case, orange, red, blue and purple masks were created. mascara_morado = cv2.inRange(hsv, morado_bajos, morado_altos).

Figure 2: HSV Model as an Inverted Pyramid.

Then, each mask experienced open and closed morphological transformations. The first transformation consists of converting to zero all pixels from the illustration that do not contain completely the structured element in its surrounding (erosion). That means, getting matrix B and matrix A, as it is demonstrated on Figure 3, they may overlap each other. If all pixels from matrix B intersects with matrix A, they will be storage as ones on a new matrix, on the contrary, they will be storage as zeros. The second step of the open transformation is dilatation. It consists in overlapping matrix B and matrix A, in difference of erosion, on this phase the first one pixel from matrix B that intersects with matrix A, will storage all ones’ pixels from matrix B on a new matrix as it is shown on Figures 4 & 5. The closed transformation applies, first, dilatation on hand of the erosion. But, it should be clear that these are not contrary operations. The combination of open and closed operations could use them for the filter and segmentation of the image. Next, OpenCV function “countNonZero” is used to follow a counting of all ones from the matrix of each color, which are storage on a variable to compare them among masks. However, the variable with higher value will indicate the color that is on observation. Ultimately, “ALTextToSpeech” was used to program the robot with the objective of saying the next phrase: “El objeto es -color-”:tts = ALProxy(‘ALTextToSpeech’, “IP”, 9559) tts.say(“El objeto es azul”) [3].

Figure 3: Erosion Example

Figure 4: Dilatation Example.

Figure 5: Color Masks taking with Nao camera. A) Original image, B) Purple Color Mask, C) Orange Color Mask, D) Masks of four colors, E) Red Color Mask and F) Blue Color Mask.

Results

The illustration process worked since figures of different colors were divided into sections and noticed them with a small error percentage, depending of the illumination, as it is shown below on Figures 6. Once a piece was located on the table, after a pair of seconds, the robot mentioned the color of the Figure. Nevertheless, there was not necessity to take screens or restart the software since recognition was executed in real time (Figure 7). Referring to the movement of figures, the goal is achieved whether they are located on the mark or near, at the same the bottle where figures are placed. This action is done in five main phases: locate hand over the figure, pick the figure, locate the figure and hand on the bottle, drop off the figure inside the bottle and lift the bottle. It is necessary to locate pieces one by one on the pointer place. Once the humanoid realizes the last movement, it synthesizes the phase: “We need to save this”. The moments are shown in Figure 8. So next we are going to show the code used to differentiate the colors using the NAO the result may vary a little bit this code is the one used for the results above (Program 1):

Figure 6: Image of Nao Humanoid Processing Color Recognition in Real Time.

Figure 7: Nao Location, Figure Marks and Bottle marked with red.

Program 1.

Figure 8: Main Moments of Figures Movement: 1) Locate hand over the Figure, 2) Pick the Figure, 3) Locate the Figure and Hand on the Bottle, 4) Drop off thefigure inside the bottle, 5) Lift the Bottle.

First the First Four Lines we Define the Libraries we are Going to use for this Program

Second we are going to define the arrays of colors to separate each color for this you need to browse in the internet for the colors you want to identify, when you have the spectrum you also need to check for the light in the environment you are at cause this may affect the way colors are seen, so a further adjustment will be needed. Make sure to put the lower spectrum of the color and the higher one. If you don’t make this it will detect colors that aren’t the ones you are asking for (Program 2). Next we are going to connect to the NAO using the proxy in it the one in the code is for the NAO used, so this may vary from NAO to NAO. Be sure to use the one in your NAo in order to connect successfully (Program 3). The line that follows is to active the superior camera depending on what you want to do you may want to use the lower camera of NAO. Then you define the resolution of the image take in mind that if you use a highest resolution the compilation of the program may take longer and have a higher chance to crash. The space for RGB is then used. We then captured the image in the selected camera. The next line [9] is makes use of the previous things defined. The variables “Ancho” and “Altura” are the ones that define the resolution being “Ancho” equal to 240 and the “Ancho” equal to 320 this is the number of pixels taken for the image so the resolution is 320x240 if higher resolution is wanted you need to adjust these valors. This also make the program slower because of the bigger array and the need to process a bigger image. Finally the image is created with that resolution.

Program 2.

Program 3.

In this part we used the command if so if there is an error or it doesn’t detect the image in the screen will appear “Cannot capture” or “Not enough data for the image” (Program 4). If the data was captured correctly the command “else” of the “if” will accommodate the binary values in an array for the format of the image. Starting from 0 giving values for “y” that is “Altura” (height) and x that is “Ancho” (width) this will accomodate de values as previously said. In this part we use the images and apply the range of colors the lower and higher spectrum defining what is considered to be the color, in a light tone or a dark one (Program 5). This part will give multiple “mascaras” this are the ones that separate each color and detect it. For this is needed the variable previously used to establish the colors ranges in the spectrum we establish the morphology with this along with the kernel that was defined for the image. This part shows 8 images one for each color used, the one for the camera that is a “normal image and one detecting all of the colors previously established showing all things that the camera of the NAO is available to detect (Program 6). Finally the command of is used again in this case to make the NAO said the color detected. In order to achieve this it’s needed to make the program recognize the highest value of the spectrum for the selected color. In the first line we can see how the color “Morado” (purple) needs to be higher than the others color like “rojo” (red), “azul” (blue) and “naranja” (orange) in order to make NAO able to differentiate from the other colors and said the name of it.

Program 4.

Program 5.

Program 6.

Figure 9: Organization of objects by size and color to support to Down Syndrome Children.

For the movement the easiest way to make it is using the software Choregraphe at first you will need to save the movements performed and latter these can be used in conjunction with another program like the color recognition one to make the NAO act in a certain way depending of the needs of the problem. For example: In order to accomplish the movement of toy pieces and storage them in a container, Choregraphe 2.1.4.14 Software was managed. To achieve this goal, the same white table was used to locate the humanoid from a distance of 5 cm between its feet and the table. On the left, the piece is located and on the right the bottle is placed. A cylinder bottle of 14 cm of height and 7.5 cm of diameter was used. A mark shown in Figure 9 was painted to point the place where pieces are located for the humanoid achieves to pick each piece and place it inside the container that also has a specific place to locate it. Choregraphe is an easy program to use the only problem that it can have is that you need to buy a license for it but depending on what you need you can use the 90-day trial to make the NAO make what you want and obtain the code you are going to need for it.

Conclusion

To conclude, due by the lack of skilled people to educate children with Down syndrome, the ninety-seven percent of those children do not receive special education they require, resulting very difficult to include them in the society. For that reason, the use of a Nao humanoid would be a great support for the education of kids with Down syndrome. Since the operation of efficient tools as tactile and precision sensors, cameras, microphone and voice synthesizer takes advantage because they attract the whole children attention. That means, an interaction between the humanoid and the user is achieved. Kids, who own Down syndrome, have short and long term deficiencies. So, activities have to be repetitive in order to accomplish the child remembers them. Meanwhile, Nao operates three learning ways: kinesthetic, visual and auditory. This research lists the first steps to support kids with Down syndrome to develop certain abilities. First, the knowledge of the four colors, which were mentioned before, and locating each piece for the robot makes its work. Also, it hopes that motor coordination would be an essential aportation for the child does the same movements from the robot. When being the child who places figures inside the container. The goal is the kid keeps this action to be applied on others objects, taking in account this will be done after any activity where several objects were operated. That is expected on a future, the humanoid programming enables an entire intersection within the infant with Down syndrome.

Future Research

Due to the lack of personnel specialized in the education of children with Down Syndrome, 97% of Down children are left without education and, therefore, the inclusion of them is difficult. Therefore, the use of the humanoid Nao would be very useful as support in the education of children with SD taking advantage of their tactile and pressure sensors, cameras, microphone and speech synthesizer can capture the attention of the child’s complete child using each of the components of the robot to achieve a humanoid-user interaction [4]. Children with DS have deficiencies in their short and long term memory, so the activities must be repetitive to get the child to retain it and with Nao the three forms of learning are used: kinesthetic [5], visual [6], and auditory [7], as is proposed. In this section, the first steps are taken to help the child develop certain skills. First of all, learning the four colors that were used in this section, followed by the collaboration, placing each piece for the humanoid to do his work [8-14]. One contribution plus motor coordination, it is expected that the child imitates the movements of the robot and is he, the one who places the pieces inside the boat, looking for this action to be recorded and can apply it in other objects, understanding that it is an action which must be carried out after any activity in which various objects were used. It is expected that in the future the programming of the humanoid will allow a total interaction with the Down child developing a complete therapy.

Public Awareness about Tetanus-https://biomedres01.blogspot.com/2021/03/public-awareness-about-tetanus.html

More BJSTR Articles : https://biomedres01.blogspot.com