Package org.apache.sysds.runtime.instructions.spark.data

Class RDDObject

java.lang.Object
org.apache.sysds.runtime.instructions.spark.data.LineageObject
org.apache.sysds.runtime.instructions.spark.data.RDDObject
public class RDDObject extends LineageObject

  • Constructor Details

    • RDDObject

      public RDDObject(org.apache.spark.api.java.JavaPairRDD<?,?> rddvar)

  • Method Details

    • getRDD

      public org.apache.spark.api.java.JavaPairRDD<?,?> getRDD()

    • setRDD

      public void setRDD(org.apache.spark.api.java.JavaPairRDD<?,?> rddHandle)

    • setCheckpointRDD

      public void setCheckpointRDD(boolean flag)

    • isCheckpointRDD

      public boolean isCheckpointRDD()

    • setHDFSFile

      public void setHDFSFile(boolean flag)

    • setHDFSFilename

      public void setHDFSFilename(String fname)

    • isHDFSFile

      public boolean isHDFSFile()

    • getHDFSFilename

      public String getHDFSFilename()

    • setParallelizedRDD

      public void setParallelizedRDD(boolean flag)

    • isParallelizedRDD

      public boolean isParallelizedRDD()

    • setPending

      public void setPending(boolean flag)

    • isPending

      public boolean isPending()

    • setDataCharacteristics

      public void setDataCharacteristics(DataCharacteristics dc)

    • getDataCharacteristics

      public DataCharacteristics getDataCharacteristics()

    • allowsShortCircuitRead

      public boolean allowsShortCircuitRead()
      Indicates if rdd is an hdfs file or a checkpoint over an hdfs file; in both cases, we can directly read the file instead of collecting the given rdd.
      Returns:
      true if rdd is an hdfs file or a checkpoint over an hdfs file

    • allowsShortCircuitCollect

      public boolean allowsShortCircuitCollect()

    • rHasCheckpointRDDChilds

      public boolean rHasCheckpointRDDChilds()