7

I am trying to write a generic scala utility function for dealing with Apache Thrift generated Java classes. All Thrift generated Java classes extend the TBase interface with the following signature:

public interface TBase<T extends TBase<?,?>, F extends TFieldIdEnum>

This definitely has always proved to be problematic when making generic thrift utility functions and in the Java world I have typically solved this by just using @SuppressWarnings("rawtypes") knowing that I was dealing with it correctly and effectively ignoring the compiler telling me it can't verify the matching types.

In the scala world, I don't seem to have the luxury or ignoring these types and have run into a situation where I can't seem to figure out how to satisfy the generic type requirements.

So I have created my own function which converts the given non-thrift object to an instance of the requested Thrift class with a signature similar to:

def myFunc[T <: TBase[T, E], E<: TFieldIdEnum](obj: Any, clazz: Class[T]): T = {

So my issue here is that to call this function I have to do something like:

myFunc[MyThriftClass,MyThriftClass._Fields]( obj, classOf[MyThriftClass] )

Without the second type parameter explicitly specified, the compilation fails with:

error: inferred type arguments [MyThriftClass,Nothing] do not conform to method myFunc's type parameter bounds [T <: org.apache.thrift.TBase[T,E],E <: org.apache.thrift.TFieldIdEnum]
    myFunc( null, classOf[MyThriftClass])
    ^
error: type mismatch;
found   : Class[MyThriftClass](classOf[MyThriftClass])
required: Class[T]
   myFunc( null, classOf[MyThriftClass])

When I pass both type parameters, everything works though I would prefer to only need to pass the class.

But what I can't seem to figure out how to get this to work if I don't know the types at compile time.

For example, thrift metadata is defined via FieldValueMetaData objects and when there's a sub-struct, the provided metadata object is StructMetaData which contains a field

public final Class<? extends TBase> structClass;

Given that this class is missing the type parameters, I don't know how to call my method with it while satisfying the generic type constraints. In the case of recursion I managed to bypass this by casting the substruct class to an instance of Class[T] which is not true because it's the parent type not the sub-type but due to type erasure it makes the compiler happy and works at runtime. But if I don't already have a generic type to cast to I don't know how to handle this aside from maybe a really hacky approach where I cast it to some fake shell of a TBase class just to make the compiler happy.

Is there a proper way to set this up to make the compiler happy with the real types? Alternatively is there a way to bypass this similarly to how I would in pure Java with @SuppressWarnings("rawtypes"). Otherwise if not then maybe I'll just have to port this logic over to Java and call it from Scala to make everything happy. Before doing this though I'd either like to learn what I'm missing or learn why what I want is not possible.

Edit:

Really what I'm looking for is some way in scala to call my existing myFunc method given a Tbase[_,_]. In Java I'd do this with suppressing raw types but in scala I'd like to either know a way to satisfy the types and allow my method to be called or a way to suppress them and allow my method to be called.

Edit 2:

Figured out a way to do what I want but still looking for cleaner solutions if they exist. What I ended up with is the following:

def myFunc[T <: TBase[_,_],T2 <: TBase[T2, E2], E2<: TFieldIdEnum](obj: Any, clazz: Class[T]): T = {
  _myFunc[T2,E2]( obj, clazz.asInstanceOf[Class[T2]] ).asInstanceOf[T]
}

def _myFunc[T <: TBase[T, E], E<: TFieldIdEnum](obj: Any, clazz: Class[T]): T = {

To be honest I don't know why it works because I haven't specified what types T2 or E2 are but this does seem to make it compile and run.

2
  • I assume that def myFunc[T <: TBase[T, _ <: TFieldIdEnum]](obj: Any, clazz: Class[T]): T doesn't work for you? But it's hard to know exactly why, or to guess what would work, because your actual problem appears to be hiding somewhere in the code you are not showing.
    – Jasper-M
    Commented Jan 15, 2020 at 13:03
  • That doesn't work because I need to actually make use of the type E so it can't simply be a wildcard. Thrift classes have methods setFieldValue and getFieldValue that take an object of type E as an argument. Unfortunately to dynamic get the TFieldIdEnums of the object, I need to use reflection as this isn't something exposed with an external generically typed method. Since I'm using reflection I actually get an object of type TFieldIdEnum instead of E and therefore I need the type E for casting purposes or else it won't let me call these methods. Commented Jan 15, 2020 at 19:18

4 Answers 4

5
+25

Short answer, that possibly can fit your purpose.

This statement is not correct.

In the scala world, I don't seem to have the luxury of ignoring these types

In the scala world, you can erase generic type parameters using forSome:

def myFunc[T <: TBase[U, E] forSome {type U; type E}, E<: TFieldIdEnum](obj: Any, clazz: Class[T])

construct T <: TBase[U, E] forSome {type U; type E} states that T is a subtype of a TBase with certain but not important type parameters.
you can shorten forSome to just _:

def myFunc[T <: TBase[_, _], E<: TFieldIdEnum](obj: Any, clazz: Class[T])

optionally, you can rid of parameter clazz by summoning implicit class tag:

def myFunc[T <: TBase[_, _]: ClassTag, E<: TFieldIdEnum](obj: Any) = {
  val c = implicitly[ClassTag[T]].runtimeClass

Much longer answer with a solution without dropping generic parameters.

In this piece of code:

def myFunc[T <: TBase[T, E], E<: TFieldIdEnum](obj: Any, clazz: Class[T]): T = ???

it seems that you trying to do kind of "type parameter currying", and then type parameter inference. The quirk is that scala does not have by-default type parameter currying in the way, that some of type parameters could be omitted and inferred by scalac. In scala you should explicitly set type parameters you want to set, and type parameters you want to be inferred from usage. This is done using auxiliary class and its companion object. Having

  def foo_1[G[_], A](fb: G[A]):G[A] = ???

Which is called:

  val v: List[Int] = ???
  foo_1[List, Int](v)

Because, you know, higher kinds cannot be that easily inferred in some situations. Then we introduce special empty case class foo[G[_]]() which is used to carry this type. We could create its instance via the apply method to have nice syntax.


  object foo{
    def apply[G[_]]: foo[G] = new foo[G]()
  }

Then we could put another apply method which will introduce second type parameter B

  case class foo[G[_]](){
    def apply[A](fa: G[A]): G[A] = foo_1[G,A](fa)
  }

Than we could have conciese boilerplateless syntax:

  val v: List[Int] = ???

  val v1 = foo_1[List, Int](v)
  val v2 = foo[List](v)
} 

Note that foo[List](v) and foo.apply[List].apply[Int](v) are just the same because of apply related syntax sugar.

The place where this method not works is where the type parameter B could not be inferred by the compiler. I hope your case is not that case.

Last, thus not least, as you haven't asked for this, but, you actually can avoid passing clazz: Class[T] explicitly as a function parameter, but instead you can turn it to an implicit parameter:

def myFunc[T <: TBase[T, E], E<: TFieldIdEnum](obj: Any)(implicit ct: ClassTag[T]): T = ???

and access it as regular variable from your function. To wrap everything up, you must pass this class tag through auxiliary class. Also, to use the trick you should erase the dependency of T on type E to allow using that via forSome.

  trait F[A, B]
  trait U

  object foo {
    def apply[T <: F[T, E] forSome {type E}]: foo[T] = new foo[T]
  }

And, to restore type E, you can ask compiler to infer such parameter E that both undegoes requirements: E <: U, and T <: F[T,E] via implicit evidence of type <:<, which generated by compiler.

  case class foo[T <: F[T, E] forSome {type E}]() {
    def apply[E <: U](a: Any)(implicit e: <:<[T, F[T, E]], ct: ClassTag[T]): T = {
      ???
      //here goes your code.
    }
  }

I hope you can use the fact T <: F[T,E] in a form of an evidence(which also subtype of T => F[T,E]).

This is how the syntax should be used:

  trait E extends U
  trait T extends F[T,E]

  foo[T].apply("abc")
1
  • So I've tried what you said here and I still can't seem to get it. The first suggestion of using TBase[_,_] seems like its not the same as using raw types but instead its the same as java wildcards. I have already tried before but the issue is that I actually need to call a method on the TBase instance that requires a parameter of type E. Using TBase[_,_] doesn't satisfy this requirement. Really what I'm looking for is some way in scala to call my existing myFunc method given a Tbase[_,_]. I tried the other approach but I still ran into the same errors so maybe i'm doing it wrong. Commented Dec 30, 2019 at 18:18
1

I would recommend using scrooge to generate your thrift classes.

It generates native scala code which is a lot nicer and more idiomatic to deal with scala, and dancing around the (mutable!!!) java classes.

Scrooge-generated structures extend ThriftStruct which is not parametrized, and solves all your problem in on shot.

3
  • I have thought about this and this is a valid suggestion but the primary reason we haven't done this is that we primarily use java and have already written a lot of utilities for thrift objects in java which would all have to be rewritten if we switched to scrooge Commented Jan 16, 2020 at 22:56
  • Then write this in java too ... Problem solved :) I am not kidding. Why not just write a java utility, and call that from scala. It'll be easier, and a lot more readable than whatever hack you come up with in scala.
    – Dima
    Commented Jan 16, 2020 at 23:41
  • so i started in scala because this particular project is scala based but after running into issues i did realize i might be better off just going back to java. however it really bugged me that i couldn't get it to work when i knew there must be a way and i posted this more out of curiosity to learn how to overcome it than for anything else. I did manage to find my hacky solution that i posted but was still curious if there's another cleaner one. Commented Jan 17, 2020 at 0:37
0

So after a lot of trial and error I discovered that this version seems to do what I need by creating new types T2 and E2 and casting to those and then casting the result back to T:

def myFunc[T <: TBase[_,_],T2 <: TBase[T2, E2], E2<: TFieldIdEnum](row: Row, clazz: Class[T]): T = {
  _myFunc[T2,E2]( row, clazz.asInstanceOf[Class[T2]] ).asInstanceOf[T]
}

def _myFunc[T <: TBase[T, E], E<: TFieldIdEnum](row: Row, clazz: Class[T]): T = {

Still looking for a cleaner solution though if it exists

0
error: inferred type arguments [MyThriftClass,Nothing] do not conform to method myFunc's type parameter bounds [T <: org.apache.thrift.TBase[T,E],E <: org.apache.thrift.TFieldIdEnum]
    myFunc( null, classOf[MyThriftClass])
    ^
error: type mismatch;
found   : Class[MyThriftClass](classOf[MyThriftClass])
required: Class[T]
   myFunc( null, classOf[MyThriftClass])

According to the error message, it's the second type parameter E that breaks the type checking. Is it necessary for myFunc? If not, you could just work without it:

def myFunc[T <: TBase[_, _]: ClassTag](obj: Any): T
1
  • Yeah unfortunately it is necessary to use the type E as i need to cast objects of TFieldIdEnum to E in order to call setFieldValue and getFieldValue on the thrift object. The reason my TFieldIdEnum are not already of type E is because i obtain them using reflection. Commented Jan 16, 2020 at 20:30

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.