1

I have designed a simple multitouch system that tracks finger location, and their status on the screen. The way that the program is designed, when two fingers are put down, they are assigned a number based on the order of placement. If finger one is lifted, finger two is still called finger two.

This code works perfectly part of the time. Meaning that I'll run my application in testing and sometimes multitouch works every time, and sometimes using more than one finger makes the application completely fail. It is irregularly reliable.

I am curious if this is a hardware error, or something about the way that I am handling touch input. The source is all available below.

Thanks for any help that you can offer!

-Nathan Tornquist

EDIT: I think that the issue may be with the way that eventListeners are registered and unregistered. Occasionally the application will crash, and work perfectly again when I reopen it.

EDIT2: The issue is simply how irregular the program is. Sometimes when I open it, it tracks the fingers perfectly. The first finger put down stays number 1, the second finger stays number 2, etc. Regardless of if you lift finger 1 after placing 2, the numbers stay assigned. Other times you'll place two fingers and which finger is 1 and which is 2 changes. The application seems to lose track of them, and the numbers switch as you leave the fingers on the screen.

EDIT3: I have tried "Touchscreen boosters" to try to try the screen to always respond correctly. It did not solve the problem.

EDIT4: After more testing, this is clearly a code error. When the application is first started, it always works perfectly. After the device is locked (when the activity is paused) and then unlocked (when the activity is resumed) multitouch stops working and I get an application that works well for single touch and is confused by multitouch.

MultitouchGameFixActivity.java

package com.nathantornquist.multitouchgame;

import android.app.Activity;
import android.content.pm.ActivityInfo;
import android.os.Bundle;
import android.view.Window;
import android.view.WindowManager;

public class MultiTouchGameFixActivity extends Activity{
    /** Called when the activity is first created. */
    MainGamePanel viewPanel;
    @Override
    public void onCreate(Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);

        //Window state functions.
        requestWindowFeature(Window.FEATURE_NO_TITLE);
        getWindow().setFlags(WindowManager.LayoutParams.FLAG_FULLSCREEN, WindowManager.LayoutParams.FLAG_FULLSCREEN);
        setRequestedOrientation(ActivityInfo.SCREEN_ORIENTATION_PORTRAIT);

        //This works without declaring a viewPanel instance here.
        //The instance declaration is needed to pass the 
        //onPause and onResume commands though.
        viewPanel = new MainGamePanel(this);
        setContentView(viewPanel);
    }

    //Restarts the accelerometer after onPause
    protected void onResume() {
        super.onResume();
        viewPanel.resume(this);

    }

    //Standard Method run when the Application loses focus.
    //This runs the pause() function in the viewPanel so that
    //the accelerometer can be paused.
    protected void onPause() {
        super.onPause();   
        viewPanel.pause();

    }

    protected void onDestroy() {
        super.onDestroy();
        viewPanel.destroy();
    }
}

MainThread.java

package com.nathantornquist.multitouchgame;

import com.nathantornquist.multitouchgame.MainGamePanel;
import android.graphics.Canvas;
import android.view.SurfaceHolder;

public class MainThread extends Thread {

private SurfaceHolder surfaceHolder;
private MainGamePanel gamePanel;
private boolean running;
public boolean pleaseWait = true;
public void setRunning(boolean running) {
    this.running = running;
}

public MainThread(SurfaceHolder surfaceHolder, MainGamePanel gamePanel) {
    super();
    this.surfaceHolder = surfaceHolder;
    this.gamePanel = gamePanel;
}

@Override
public void run() 
{
    Canvas canvas;
    while (running) {
        if(!pleaseWait) {
            canvas = null;
            // try locking the canvas for exclusive pixel editing on the surface
            try {
                canvas = this.surfaceHolder.lockCanvas();
                synchronized (surfaceHolder) {
                    // update game state
                    this.gamePanel.update();

                    // draws the canvas on the panel
                    this.gamePanel.onDraw(canvas);
                }
            } finally {
                // in case of an exception the surface is not left in
                // an inconsistent state
                if (canvas != null) {
                    surfaceHolder.unlockCanvasAndPost(canvas);
                }
            }   // end finally            
        }
        else {
            synchronized (this) {
                try {
                    wait();
                } catch (Exception e) { }
            }
        }
    }
}
}

MainGamePanel.java

package com.nathantornquist.multitouchgame;

import android.R.string;
import android.app.Activity;
import android.content.Context;
import android.graphics.Canvas;
import android.graphics.Color;
import android.graphics.Paint;
import android.graphics.Paint.Style;
import android.hardware.Sensor;
import android.hardware.SensorEvent;
import android.hardware.SensorEventListener;
import android.hardware.SensorManager;
import android.util.Log;
import android.view.Display;
import android.view.MotionEvent;
import android.view.SurfaceHolder;
import android.view.SurfaceView;

public class MainGamePanel extends SurfaceView implements SensorEventListener, SurfaceHolder.Callback 
{
    //Variable Declarations.
    private MainThread thread;

    public int screenWidth;
    public int screenHeight;

    private SensorManager mSensorManager;
    private Sensor mAccelerometer;    

    Paint paint;
    public int fingerOneDown;
    public int fingerTwoDown;
    public int fingerThreeDown;
    public int fingerFourDown;
    public float fingerOneX;
    public float fingerOneY;
    public float fingerTwoX;
    public float fingerTwoY;
    public float fingerThreeX;
    public float fingerThreeY;
    public float fingerFourX;
    public float fingerFourY;

    public MainGamePanel(Context context)
    {
        super(context);

        getHolder().addCallback(this);

        thread = new MainThread(getHolder(),this);

        paint = new Paint();
        paint.setAntiAlias(true);

        Display display = ((Activity) context).getWindowManager().getDefaultDisplay(); 
        screenWidth = display.getWidth();
        screenHeight = display.getHeight();

        mSensorManager = (SensorManager) context.getSystemService(Context.SENSOR_SERVICE);
        mAccelerometer = mSensorManager.getDefaultSensor(Sensor.TYPE_ACCELEROMETER);
        mSensorManager.registerListener(this, mAccelerometer, SensorManager.SENSOR_DELAY_FASTEST);

        fingerOneDown = 0;
        fingerTwoDown = 0;
        fingerThreeDown = 0;
        fingerFourDown = 0;
        fingerOneX = 0;
        fingerOneY = 0;
        fingerTwoX = 0;
        fingerTwoY = 0;
        fingerThreeX = 0;
        fingerThreeY = 0;
        fingerFourX = 0;
        fingerFourY = 0;

        setFocusable(true);

        thread.setRunning(true);
        thread.start();

    }

    @Override
    public void surfaceChanged(SurfaceHolder holder, int format, int width,
            int height) {
    }

    @Override
    public void surfaceCreated(SurfaceHolder holder) {
        //continue the thread
        synchronized (thread) {
            thread.pleaseWait = false;
            thread.notifyAll();
        }

    }

    @Override
    public void surfaceDestroyed(SurfaceHolder holder) {
        //pause the thread
        synchronized (thread) {
            thread.pleaseWait = true;
        }

    }

    @Override
    public boolean onTouchEvent(MotionEvent event) {
        int action = event.getAction() & MotionEvent.ACTION_MASK;
        int pointerIndex = (event.getAction() & MotionEvent.ACTION_POINTER_ID_MASK) >> MotionEvent.ACTION_POINTER_ID_SHIFT;
        int pointerId = event.getPointerId(pointerIndex);
        switch (action) {
        case MotionEvent.ACTION_DOWN:
        case MotionEvent.ACTION_POINTER_DOWN:
            Log.d("pointer id - down",Integer.toString(pointerId));
            if (pointerId == 0)
            {
                fingerOneDown = 1;
                fingerOneX = event.getX(pointerIndex);
                fingerOneY = event.getY(pointerIndex);
            }
            if (pointerId == 1)
            {
                fingerTwoDown = 1;
                fingerTwoX = event.getX(pointerIndex);
                fingerTwoY = event.getY(pointerIndex);
            }
            if(pointerId == 2)
            {
                fingerThreeDown = 1;
                fingerThreeX = event.getX(pointerIndex);
                fingerThreeY = event.getY(pointerIndex);
            }
            if(pointerId == 3)
            {
                fingerFourDown = 1;
                fingerFourX = event.getX(pointerIndex);
                fingerFourY = event.getY(pointerIndex);
            }
            break;

        case MotionEvent.ACTION_UP:          
        case MotionEvent.ACTION_POINTER_UP:
        case MotionEvent.ACTION_CANCEL:
            Log.d("pointer id - cancel",Integer.toString(pointerId));
            if (pointerId == 0)
            {
                fingerOneDown = 0;
                fingerOneX = event.getX(pointerIndex);
                fingerOneY = event.getY(pointerIndex);
            }
            if (pointerId == 1)
            {
                fingerTwoDown = 0;
                fingerTwoX = event.getX(pointerIndex);
                fingerTwoY = event.getY(pointerIndex);
            }
            if(pointerId == 2)
            {
                fingerThreeDown = 0;
                fingerThreeX = event.getX(pointerIndex);
                fingerThreeY = event.getY(pointerIndex);
            }
            if(pointerId == 3)
            {
                fingerFourDown = 0;
                fingerFourX = event.getX(pointerIndex);
                fingerFourY = event.getY(pointerIndex);
            }
            break;

        case MotionEvent.ACTION_MOVE:

            int pointerCount = event.getPointerCount();
            for(int i = 0; i < pointerCount; ++i)
            {
                pointerIndex = i;
                pointerId = event.getPointerId(pointerIndex);
                Log.d("pointer id - move",Integer.toString(pointerId));
                if(pointerId == 0)
                {
                    fingerOneDown = 1;
                    fingerOneX = event.getX(pointerIndex);
                    fingerOneY = event.getY(pointerIndex);
                }
                if(pointerId == 1)
                {
                    fingerTwoDown = 1;
                    fingerTwoX = event.getX(pointerIndex);
                    fingerTwoY = event.getY(pointerIndex);
                }
                if(pointerId == 2)
                {
                    fingerThreeDown = 1;
                    fingerThreeX = event.getX(pointerIndex);
                    fingerThreeY = event.getY(pointerIndex);
                }
                if(pointerId == 3)
                {
                    fingerFourDown = 1;
                    fingerFourX = event.getX(pointerIndex);
                    fingerFourY = event.getY(pointerIndex);
                }
            }
            break;
        }
        return true;
    }

    @Override
    protected void onDraw(Canvas canvas) 
    {        
        paint.setColor(Color.WHITE); 
        paint.setStyle(Style.FILL); 
        canvas.drawPaint(paint); 

        if (fingerOneDown == 1)
        {
            paint.setColor(Color.BLUE); 
            paint.setTextSize(20); 
            canvas.drawText("1", fingerOneX, fingerOneY - 30, paint); 
        }

        if (fingerTwoDown == 1)
        {
            paint.setColor(Color.RED); 
            paint.setTextSize(20); 
            canvas.drawText("2", fingerTwoX, fingerTwoY - 30, paint); 
        }
        if (fingerThreeDown == 1)
        {
            paint.setColor(Color.GREEN); 
            paint.setTextSize(20); 
            canvas.drawText("3", fingerThreeX, fingerThreeY - 30, paint); 
        }
        if (fingerFourDown == 1)
        {
            paint.setColor(Color.BLACK); 
            paint.setTextSize(20); 
            canvas.drawText("4", fingerFourX, fingerFourY - 30, paint); 
        }
    }

    public void update() {
    }

    @Override
    public void onAccuracyChanged(Sensor sensor, int accuracy) {        
    }

    @Override
    public void onSensorChanged(SensorEvent event) {

    }

    public void pause() {
        mSensorManager.unregisterListener(this);
    }

    public void resume(Context context) {
        mSensorManager.registerListener(this, mAccelerometer, SensorManager.SENSOR_DELAY_FASTEST);      
    }

    public void destroy() {
        thread.setRunning(false);

        if (thread != null)
        {
            Thread killThread = thread;
            thread = null;
            killThread.interrupt();
        }   

    }


}
2
  • A little detail on what you mean by 'the application fails' would be nice. Does it crash? If so, what is the stack trace? Does it simply not stop functioning properly? If so, how?
    – Zarjio
    Mar 24, 2012 at 14:55
  • I added more information above. See EDIT2. I hope that adds come clarity. Mar 24, 2012 at 15:17

2 Answers 2

0

I'll throw out some ideas, even though my android skills are a bit lacking...

How are you testing this? On a real device I guess? Can this be tested on the emulator? It seems like it would be prone to hardware issues.

If the there is a glitch and the screen looses track of the fingers even just for a millisec, wont it start over with indexing the fingers? If that is the case it would number the fingers pretty randomly causing the issues you are experiencing.

Some ideas:

  • Dont track the index of fingers, just keep list of the fingers and their relative positions. But I guess part of the game requires you to keep an index of each finger.
  • Log if you loose connection with a finger, that will show if this is an isse. I guess MotionEvent.ACTION_POINTER_UP is the event you should log then. In fact, I'd add a log for each event possible just to see what is happening.
  • Add a method that tries to determine which finger has been placed on the screen, i.e it compares the current position of the finger with the known values of previous fingers and if its within a threshold, assume its the same finger. Probably want to add a timer to "forget" previous fingers aswell.

I guess you have been looking at the multitouch example on Android developer blog to make sure you are using the APIs correctly?

Hope it may give you some help.

1
  • There is no way to test this with an emulator, because of the multitouch nature. Thanks for your response though. The hardest part of testing this is knowing what causes the events. I feel like it is a code error, but any weird errors that I generate could be caused by bad code on my.part instead of faulty hardware. It is a difficult issue. Mar 29, 2012 at 19:56
0

The problem is solved by disabling WidgetLocker. That application seems to mess up some programs occasionally. I've noticed issues in other applications as well when I lock the screen while playing a game.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.