mithrix February 2016

How do I calculate Spark Statistics that are not of type Double

The spark documenation includes tools for calculating min, max, mean statistics on values of type "double" but how do you handle a spark/java/cassandra scenario when trying to handle column values of type float?

EDITED To Show Resolution:

import org.apache.spark.sql.DataFrame;
import static org.apache.spark.sql.functions.*;

DataFrame df = sqlContext.read()
        .format("org.apache.spark.sql.cassandra")
        .option("table",  "someTable")
        .option("keyspace", "someKeyspace")
        .load();

df.groupBy(col("keyColumn"))
        .agg(min("valueColumn"), max("valueColumn"), avg("valueColumn"))
        .show();

Answers


AndrewIsOffline February 2016

Cast it. (double) Variable_here will be the variable's value, but as a double.

Post Status

Asked in February 2016
Viewed 1,316 times
Voted 7
Answered 1 times

Search




Leave an answer