L
- The type of the labels in the DatasetF
- The type of the features in the Datasetpublic class RVFDataset<L,F> extends GeneralDataset<L,F>
ClassifierFactory
that incrementally builds
a more memory-efficient representation of a List
of RVFDatum
objects for the purposes of training a Classifier
with a
ClassifierFactory
.data, featureIndex, labelIndex, labels, size
Constructor and Description |
---|
RVFDataset() |
RVFDataset(Index<F> featureIndex,
Index<L> labelIndex) |
RVFDataset(Index<L> labelIndex,
int[] labels,
Index<F> featureIndex,
int[][] data,
double[][] values)
Constructor that fully specifies a Dataset.
|
RVFDataset(int numDatums) |
RVFDataset(int numDatums,
Index<F> featureIndex,
Index<L> labelIndex) |
Modifier and Type | Method and Description |
---|---|
void |
add(Datum<L,F> d) |
void |
add(Datum<L,F> d,
java.lang.String src,
java.lang.String id) |
void |
applyFeatureCountThreshold(int k)
Applies a feature count threshold to the RVFDataset.
|
void |
applyFeatureMaxCountThreshold(int k)
Applies a feature max count threshold to the RVFDataset.
|
void |
clear()
Resets the Dataset so that it is empty and ready to collect data.
|
void |
clear(int numDatums)
Resets the Dataset so that it is empty and ready to collect data.
|
void |
ensureRealValues()
Checks if the dataset has any unbounded values.
|
RVFDatum<L,F> |
getDatum(int index) |
RVFDatum<L,F> |
getRVFDatum(int index) |
java.lang.String |
getRVFDatumId(int index) |
java.lang.String |
getRVFDatumSource(int index) |
double[][] |
getValuesArray() |
protected void |
initialize(int numDatums)
This method takes care of resetting values of the dataset
such that it is empty with an initial capacity of numDatums.
|
java.util.Iterator<RVFDatum<L,F>> |
iterator() |
static void |
main(java.lang.String[] args) |
void |
printFullFeatureMatrix(java.io.PrintWriter pw)
prints the full feature matrix in tab-delimited form.
|
void |
printFullFeatureMatrixWithValues(java.io.PrintWriter pw)
Modification of printFullFeatureMatrix to correct bugs & print values
(Rajat).
|
void |
printSparseFeatureMatrix()
Prints the sparse feature matrix using
printSparseFeatureMatrix(PrintWriter) to System.out . |
void |
printSparseFeatureMatrix(java.io.PrintWriter pw)
Prints a sparse feature matrix representation of the Dataset.
|
void |
printSparseFeatureValues(int datumNo,
java.io.PrintWriter pw)
Prints a sparse feature-value output of the Dataset.
|
void |
printSparseFeatureValues(java.io.PrintWriter pw)
Prints a sparse feature-value output of the Dataset.
|
void |
randomize(int randomSeed)
Randomizes the data array in place.
|
void |
readSVMLightFormat(java.io.File file)
Read SVM-light formatted data into this dataset.
|
static RVFDataset<java.lang.String,java.lang.String> |
readSVMLightFormat(java.lang.String filename)
Constructs a Dataset by reading in a file in SVM light format.
|
static RVFDataset<java.lang.String,java.lang.String> |
readSVMLightFormat(java.lang.String filename,
Index<java.lang.String> featureIndex,
Index<java.lang.String> labelIndex)
Constructs a Dataset by reading in a file in SVM light format.
|
static RVFDataset<java.lang.String,java.lang.String> |
readSVMLightFormat(java.lang.String filename,
java.util.List<java.lang.String> lines)
Constructs a Dataset by reading in a file in SVM light format.
|
RVFDataset<L,F> |
scaleDataset(RVFDataset<L,F> dataset)
Scales the values of each feature in each linearly using the min and max
values found in the training set.
|
RVFDataset<L,F> |
scaleDatasetGaussian(RVFDataset<L,F> dataset) |
RVFDatum<L,F> |
scaleDatum(RVFDatum<L,F> datum)
Scales the values of each feature linearly using the min and max values
found in the training set.
|
RVFDatum<L,F> |
scaleDatumGaussian(RVFDatum<L,F> datum) |
void |
scaleFeatures()
Scales feature values linearly such that each feature value lies between 0
and 1.
|
void |
scaleFeaturesGaussian() |
void |
selectFeaturesFromSet(java.util.Set<F> featureSet)
Removes all features from the dataset that are not in featureSet.
|
Pair<GeneralDataset<L,F>,GeneralDataset<L,F>> |
split(double percentDev) |
Pair<GeneralDataset<L,F>,GeneralDataset<L,F>> |
split(int start,
int end) |
void |
summaryStatistics()
Prints some summary statistics to stderr for the Dataset.
|
static RVFDatum<java.lang.String,java.lang.String> |
svmLightLineToRVFDatum(java.lang.String l) |
java.lang.String |
toString() |
java.lang.String |
toSummaryString() |
void |
writeSVMLightFormat(java.io.File file)
Write the dataset in SVM-light format to the file.
|
void |
writeSVMLightFormat(java.io.PrintWriter writer) |
addAll, featureIndex, getDataArray, getFeatureCounts, getLabelsArray, labelIndex, labelIterator, makeSvmLabelMap, mapDataset, mapDataset, mapDatum, numClasses, numDatumsPerLabel, numFeatures, numFeatureTokens, numFeatureTypes, printSVMLightFormat, printSVMLightFormat, sampleDataset, size, trimData, trimLabels, trimToSize, trimToSize, trimToSize
public Pair<GeneralDataset<L,F>,GeneralDataset<L,F>> split(double percentDev)
split
in class GeneralDataset<L,F>
public void scaleFeaturesGaussian()
public void scaleFeatures()
public void ensureRealValues()
public RVFDataset<L,F> scaleDataset(RVFDataset<L,F> dataset)
dataset
- public RVFDatum<L,F> scaleDatum(RVFDatum<L,F> datum)
datum
- public RVFDataset<L,F> scaleDatasetGaussian(RVFDataset<L,F> dataset)
public Pair<GeneralDataset<L,F>,GeneralDataset<L,F>> split(int start, int end)
split
in class GeneralDataset<L,F>
public RVFDatum<L,F> getDatum(int index)
getDatum
in class GeneralDataset<L,F>
public RVFDatum<L,F> getRVFDatum(int index)
getRVFDatum
in class GeneralDataset<L,F>
public java.lang.String getRVFDatumSource(int index)
public java.lang.String getRVFDatumId(int index)
public void clear()
clear
in class GeneralDataset<L,F>
public void clear(int numDatums)
clear
in class GeneralDataset<L,F>
numDatums
- initial capacity of datasetprotected void initialize(int numDatums)
GeneralDataset
initialize
in class GeneralDataset<L,F>
numDatums
- initial capacity of datasetpublic void summaryStatistics()
summaryStatistics
in class GeneralDataset<L,F>
public void printFullFeatureMatrix(java.io.PrintWriter pw)
public void printFullFeatureMatrixWithValues(java.io.PrintWriter pw)
public static RVFDataset<java.lang.String,java.lang.String> readSVMLightFormat(java.lang.String filename)
public static RVFDataset<java.lang.String,java.lang.String> readSVMLightFormat(java.lang.String filename, java.util.List<java.lang.String> lines)
public static RVFDataset<java.lang.String,java.lang.String> readSVMLightFormat(java.lang.String filename, Index<java.lang.String> featureIndex, Index<java.lang.String> labelIndex)
public void selectFeaturesFromSet(java.util.Set<F> featureSet)
featureSet
- public void applyFeatureCountThreshold(int k)
applyFeatureCountThreshold
in class GeneralDataset<L,F>
public void applyFeatureMaxCountThreshold(int k)
applyFeatureMaxCountThreshold
in class GeneralDataset<L,F>
public static RVFDatum<java.lang.String,java.lang.String> svmLightLineToRVFDatum(java.lang.String l)
public void readSVMLightFormat(java.io.File file)
file
- The file from which the data should be read.public void writeSVMLightFormat(java.io.File file) throws java.io.FileNotFoundException
readSVMLightFormat(File)
.file
- The location where the dataset should be written.java.io.FileNotFoundException
public void writeSVMLightFormat(java.io.PrintWriter writer)
public void printSparseFeatureMatrix()
printSparseFeatureMatrix(PrintWriter)
to System.out
.public void printSparseFeatureMatrix(java.io.PrintWriter pw)
Object.toString()
representations of features.public void printSparseFeatureValues(java.io.PrintWriter pw)
Object.toString()
representations of features. This is probably
what you want for RVFDataset since the above two methods seem useless and
unused.public void printSparseFeatureValues(int datumNo, java.io.PrintWriter pw)
Object.toString()
representations of features. This is probably
what you want for RVFDataset since the above two methods seem useless and
unused.public static void main(java.lang.String[] args)
public double[][] getValuesArray()
getValuesArray
in class GeneralDataset<L,F>
public java.lang.String toString()
toString
in class java.lang.Object
public java.lang.String toSummaryString()
public void randomize(int randomSeed)
randomize
in class GeneralDataset<L,F>