FrontPage

What's this?

  • This product is "Auto Chasing Turtle".
  • By autonomous control, this robot recognizes people's face and approaches to the detected human. The scene that it is working in real time can be seen by iPad. It is built by using Kinect as sensor and using Linaro kernel + Android + openFrameworks as application framework.
  • YouTube Video
DSC_0012.jpg
DSC_0001.jpg
DSC_0006.jpg

The thing to prepare

Reference data

download

license

  • GPL2.0

slide

Speech&Award

Linaro Developer Summit 11/05

  • We speech.

Laval Virtual 2011

  • We are nominated.

Campus Party 2011 Columbia

  • We speech.

Detail explanation for source code

Hardware

  • Connect all hardwares.
  1. battery
  2. beagleboard-xM
  3. Kinect
  4. WiFiRouter

Software

What doing?

  1. Take the RGB camera's image
    1. Save this image as jpeg
  2. Try to recognize face
    1. If fail, search random.
  3. Calculate the course which it should follow.
    1. Move kinect's Angle
  4. Calculate the distance to target

Detect Face

  • Take the RGB camera's image
    • Take from Kinect's RGB camera by ofxDroidKinect.
      • testApp.cpp : 36 line
        void testApp::draw() {
           kinect.draw(0, 0, 480, 320);
           flame++;
           if (flame > 5) {
                flame = 0;
                colorImg->setFromPixels(kinect.getPixels(), kinect.width, kinect.height);
           }
        }
  • Save this image as jpeg
    • Android's FaceDetector class can't detect by the Kinect's Bitmap format. Therefore change format Bitmap to JPEG.
      • OFActivity.java : 53-72 line
        Bitmap bitmap = Bitmap.createBitmap(w, h, Bitmap.Config.ARGB_8888);
        byte[] pixels = OFAndroid.getImgPixels();
        
        if (pixels != null) {
            for (int i=0;i<w;i++) {  
                for (int j=0;j<h;j++){
                    int index = (j * w + i) * 3;
                    bitmap.setPixel(i, j, Color.rgb(pixels[index++], pixels[index++], pixels[index]));
                }
           }
           
            try {
               // save adcard 
                FileOutputStream fos = new FileOutputStream("/sdcard/screenshot.jpg");
                bitmap.compress(Bitmap.CompressFormat.JPEG, 100, fos);
                fos.flush();
                fos.close();
            } catch (Exception e) {
                // check exception
            }
  • Try to recognize face
    • Use Android's FaceDetector class
      • OFActivity.java : 77-80 line
            FaceDetector.Face[] faces = new FaceDetector.Face[1];
            if (bitmap != null) {
                FaceDetector detector = new FaceDetector(w, h, faces.length);
                int numFaces = detector.findFaces(bitmap, faces);
  • If fail, search random.
      • OFActivity.java : 138-144 line
        } else {
            /*
             * search turn
             */ 
            Random random = new Random();
            int res = random.nextInt(3);
            if (res == 0)			DroidBot.getInstance().turnRight2();
            else if (res == 1)		DroidBot.getInstance().turnRight4();
            else if (res == 2)		DroidBot.getInstance().turnLeft2();
            else if (res == 3)		DroidBot.getInstance().turnLeft4();
        }

Calculate Course & Destance

  • Calculate the course which it should follow.
    • RGB image is divided into 4 pieces at a width direction. And the course is diceded by Face position.
      • OFActivity.java : 103-109 line
        if (pointX > 0 && pointX < w/4) {
            DroidBot.getInstance().turnRight2();    // right position
        } else if (pointX >= w/4 && pointX <= 3*w/4) {
            ;                                       // center position
        } else if (pointX > 3*w/4 && pointX <= w) {
            DroidBot.getInstance().turnLeft2();     // left position
        }
  • Move kinect's Angle
    • The maximum kinect's Angle is "30". And the kinect's Angle is diceded on the basis of it by Face position.
      • OFActivity.java : 117-119 line
        int angle = 30 - pointY*30/h;
        if (angle > 0 && angle <= 30)
            OFAndroid.setAngle(angle);
  • Calculate the distance to target
    • Get from Kinect's Z(depth) camera by ofxDroidKinect.
    • The kinect's Angle is diceded on the basis of it by Face position.
      • OFActivity.java : 127-132 line
        int dist = OFAndroid.getDistance(pointX, pointY);
        if (dist < 100)                     DroidBot.getInstance().walkBack4();
        else if (dist >= 100 && dist < 150) DroidBot.getInstance().walkToward4();
        else if (dist >= 150 && dist < 200) DroidBot.getInstance().walkToward8();
        else if (dist >= 200 && dist < 300) DroidBot.getInstance().walkToward16();
        else if (dist >= 300)               DroidBot.getInstance().walkToward32();

Control Robot

  • KONDO Animal is controled by serial.
  • KONDO Animal uses RCB-3 control unit. and send to control command by serial.

Connect iPad

  • iPad viewer is VNC client.
    • Using Android VNC Server.
      • init.rc
        service vncserver /data/androidvncserver -k /dev/input/event0 -t /dev/input/event0

Contact us

  • info @ siprop.org

At the last

  • If this is helpful to you, Please donate for "2011 Japanese Earthquake and Tsunami".
  • We are Japanese community team. We pray Japan revives.

Information

Donation

Contact us

  • info
    • info atmark siprop.org
  • Noritsuna
    • noritsuna atmark siprop.org