0

I'm working on an App that has objects that must be available to all instances but also have synchronized access for certain methods within the object.

For instance I have this object:

public class PlanetID implements Serializable {
    public PlanetID() {
        id = 0;
    }

    public long generateID() {
        id++;
        return id;
    }

    private long id;
}

It's a simple object that creates a long (id) in series. It's necessary that this object generate a unique id every time. At the moment I have a static synchronized method that handles the Datastore access and storage along with the MemCache access and storage. It works for this particular method but I can already see issues with more complex objects that require a user to be able to access non-synchronized variables along with synchronized variables.

Is there some way to make an object global and allow for both synchronized methods and non-synchronized methods along with the storage of the object when those synchronized objects are accessed?

EDIT: I think people focused too much on the example I gave them and not on the bigger question of having a global variable which can be accessed by all instances and having synchronized access to specific methods while allowing asynchronous access to others.

Here's a better example in hopes it makes things a big more clearer.

Ex.

public class Market implements Serializable {
public Market() {
    mineral1 = new ArrayList<Listing>();
    mineral2 = new ArrayList<Listing>();
    mineral3 = new ArrayList<Listing>();
    mineral4 = new ArrayList<Listing>();
}

public void addListing(int mineral, String userID, int price, long amount) { //Doesn't require synchronized access
    switch (mineral) {
    case MINERAL1:
        mineral1.add(new Listing(userID, price, amount));
        break;
    case MINERAL2:
        mineral2.add(new Listing(userID, price, amount));
        break;
    case MINERAL3:
        mineral3.add(new Listing(userID, price, amount));
        break;
    case MINERAL4:
        mineral4.add(new Listing(userID, price, amount));
        break;
    }
}

public void purchased(int mineral, String userID, long amount) { //Requires synchronized access
    ArrayList<Listing> mineralList = null;

    switch (mineral) {
    case MINERAL1:
        mineralList = mineral1;
        break;
    case MINERAL2:
        mineralList = mineral2;
        break;
    case MINERAL3:
        mineralList = mineral3;
        break;
    case MINERAL4:
        mineralList = mineral4;
        break;
    }       

    Listing remove = null;
    for (Listing listing : mineralList)
        if (listing.userID == userID)
            if (listing.amount > amount) {
                listing.amount -= amount;
                return;
            } else{
                remove = listing;
                break;
            }

    mineralList.remove(remove);
            Collections.sort(mineralList);
}

public JSONObject toJSON(int mineral) { //Does not require synchronized access
    JSONObject jsonObject = new JSONObject();

    try {
        switch (mineral) {
        case MINERAL1:
            for (Listing listing : mineral1)
                jsonObject.accumulate(Player.MINERAL1, listing.toJSON());
            break;
        case MINERAL2:
            for (Listing listing : mineral2)
                jsonObject.accumulate(Player.MINERAL2, listing.toJSON());
            break;
        case MINERAL3:
            for (Listing listing : mineral3)
                jsonObject.accumulate(Player.MINERAL3, listing.toJSON());
            break;
        case MINERAL4:
            for (Listing listing : mineral4)
                jsonObject.accumulate(Player.MINERAL4, listing.toJSON());
            break;
        }
    } catch (JSONException e) {

    }

    return jsonObject;
}

public static final int MINERAL1 = 0;
public static final int MINERAL2 = 1;
public static final int MINERAL3 = 2;
public static final int MINERAL4 = 3;

private ArrayList<Listing> mineral1;
private ArrayList<Listing> mineral2;
private ArrayList<Listing> mineral3;
private ArrayList<Listing> mineral4;

private class Listing implements Serializable, Comparable<Listing> {
    public Listing(String userID, int price, long amount) {
        this.userID = userID;
        this.price = price;
        this.amount = amount;
    }

    public JSONObject toJSON() {
        JSONObject jsonObject = new JSONObject();

        try {
            jsonObject.put("UserID", userID);
            jsonObject.put("Price", price);
            jsonObject.put("Amount", amount);
        } catch (JSONException e) {
            // TODO Auto-generated catch block
            e.printStackTrace();
        }

        return jsonObject;
    }

    @Override
    public int compareTo(Listing listing) {
        return (price < listing.price ? -1 : (price == listing.price ? 0 : 1));
    }

    public String userID;
    public int price;
    public long amount;
}

}

1
  • Would the transaction prevent the same ID being given more than once if the object is being accessed from memcache? Also, does a transaction prevent access to an object while being made? Apr 12, 2012 at 5:51

5 Answers 5

1

With GAE, the Java language is NOT going to hide all the datastore abstractions for you.

Stop thinking in terms of global variables and methods. These are Java language constructs. Start thinking in terms of datastore constructs - entities, datastore accesses, and transactions.

On GAE, your code will be simultaneously running on many servers, they will not share global variables, the "shared data" is in the datastore (or memcache)

An entity is an object in the datastore. You can make datastore fetches from anywhere in your code, so they can replace your global variables. You define transactions within your methods to synchronize datastore accesses and ensure a transaction only happens once. You can use transactions in some methods, and don't use transactions when you don't need synchronization.

You shouldn't need your global ArrayLists of minerals. When you handle a purchase, you essentially need a transaction where you fetch a listing from the datastore, or create it if it doesn't exist, update the user, and write it back to the datastore. You probably want to read up on the datastore before continuing.

5
  • I have the PlanetID object being fetched from the datastore and then placed into memcache from within a static method which manages access to PlanetID. The static method then stores the updated object serialized into the datastore within an entity and then back into the memcache. This is my hacked way of maintaining synchronization at the moment. The need to generate a unique ID requires a global object to maintain. Rather than reading and writing constantly to and from an entity I have an object which is cached. Apr 12, 2012 at 15:02
  • I should add that I understand what you mean about thinking in terms of global variables, I'm simply using the language to convey an idea. I use Memcache to make my object global as memcache is shared by all instances. Apr 12, 2012 at 15:09
  • Do you have multiple planets? If so each planet should have its own entity in the datastore. You can use memcache as a cache instead of hitting the datastore, but you cannot use it to replace the datastore - it's possible for memcache data to disappear at any time. Figure out how you would store your various data types (planets, minerals, listings) in the datastore, rather than as Java classes. The solution should be much clearer when you look at it that way. Treating the datastore as a bunch of global variables doesn't help with understanding.
    – dragonx
    Apr 12, 2012 at 15:50
  • At the moment I have an entity for each player and the player object contains planets, minerals, and various other data structures. The player object is then serialized and placed in a Blob and then placed into the entity for data storage. I found it to be a much cleaner solution but I guess it doesn't play well with GAE. Apr 12, 2012 at 20:53
  • Your design sounds great if you were running on a single system; the architecture doesn't translate well to a massively scalable web architecture. You'd most likely want to have a separate entity for each instance of a planet, mineral or whatever datastructure. If a planet contains a mineral for example, you'd store a key to the mineral in the planet, then you can fetch the planet, then fetch the mineral. For synchonized access to say a mineral, you'd have a transaction in which you fetch, modify, and save a given mineral.
    – dragonx
    Apr 12, 2012 at 21:39
0

Other approach beside transaction is to use a single backend instance to keep your global object, and have all access to the object synchronized there. All other instances need to access this backend instance using URLFetch to get the state of the object.

This is a horrible performance bottleneck though if your app want to scale up smoothly, please don't use it, I'm just pointing out alternative approaches. In fact, if possible, please kindly avoid the need to have a synchronized global object on a distributed application in the first place.

3
  • Unfortunately I can't see a way to avoid this. I need to generate a unique ID every time and would prefer in sequence. My question didn't use the best example. I will have at least 2 other global objects, one being a Market to allow the purchase of virtual goods (with virtual money of course) and I need to prevent double purchases but also allow access to the listings. Apr 12, 2012 at 11:52
  • I should also add that if it becomes too much of a bottleneck I will remove the datastore write after every update and have a cron save the objects every x minutes to improve performance. Apr 12, 2012 at 11:56
  • @AlejandroHuerta: First, preventing double purchases could be done in other, much easier, way than having a synchronized method. You could use a TaskQueue with retry set to 1 so that failed purchases would not be re-tried. Second, why do you need the ID to be in sequence? If it's for sorting purpose, use a sortable value in one of the values for the kind. Third, how critical would the sorting be in your case? Timestamps generally have desynchronizations in of several seconds between instance servers, if some slight chance of misordering is acceptable, use timestamp for your sortable value. Apr 13, 2012 at 7:31
0

Have a look at DatastoreService.allocateIds - it doesn't matter whether or not you're actually using the generated ID to write a datastore entity, you can get unique long numbers out of this in an efficient manner.

Please note however that they're not guaranteed to be in sequence, they're only guaranteed to be unique - the question doesn't state being sequential as a requirement.

public class PlanetID implements Serializable
{
    private DatastoreService ds;

    public PlanetID()
    {
        ds = DatastoreServiceFactory.getDatastoreService();
    }

    public long generateID()
    {
        return ds.allocateIds("Kind_That_Will_Never_Be_Used", 1L).getStart().getId();
    }
}
1
  • 1
    Unfortunately, in the comments for my question, the asker seems to prefer the ID to be in sequence, which brings a boatload of new headaches to the problem. Apr 13, 2012 at 7:23
0

As noted in comments - you can use transactions to achieve this:

  1. Start a transaction
  2. Create or update the SomeObject entity with index and someText properties.
  3. Commit transaction. If two instances are trying to do this simultaneously then one will get an exception and will need to retry (get the data again, increment, put all in transaction).

Edit:

(removed section on sharded counters as they do not guarantee )

Note that above solution has a write bottleneck at about 1 write/s. If you need a higher performance solution you could look into using backend instances.

3
  • Looking at his code I assumed he basically wants a counter: getTick() and index++. Apr 12, 2012 at 10:04
  • It seems my example is confusing people and didn't quiet get the question across so I am updating it with one of the more complex objects that I want to actually use but falls under the same requirements. Apr 12, 2012 at 11:59
  • He said "It's a simple object that creates a long (id) in series. It's necessary that this object generate a unique id every time." - you can't do that with a sharded counter. Apr 12, 2012 at 12:11
0

Whilst technically possible, you should heed the advice of other answers and use the services supplied by Google, such as datastore and memcache.

However, you could use a single backend which contains your data, then use your favourite RPC method to read and write data into the shared object. You will need to be aware that although it doesn't happen often, backends are not guaranteed not to die randomly - so you could lose all the data in this object.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.