edu.stanford.nlp.tmt.model.lda

GibbsLDA

class GibbsLDA extends LDA[HardAssignmentModelState, GibbsLDADocument, (String, Array[Int])] with HardAssignmentModel[LDAModelParams, LDADocumentParams, GibbsLDADocument]

Collapsed Gibbs sampler for LDA learning and inference. This class is not threadsafe for learning. However, it is threadsafe for inerence, but no guarantees are provided about repeatability in a threaded environment if the number of threads is different between runs. This is because each thread is given its own random number generator to avoid synchronization overhead, so the sequence of random numbers seen on an particular document may be a function of the number of threads.

Linear Supertypes
Ordering
  1. Alphabetic
  2. By inheritance
Inherited
  1. Hide All
  2. Show all
  1. GibbsLDA
  2. HardAssignmentModel
  3. LDA
  4. DirichletTopicSmoothing
  5. DirichletTermSmoothing
  6. ClosedTopicSet
  7. TopicModel
  8. RepCheck
  9. Stateful
  10. AnyRef
  11. Any
Visibility
  1. Public
  2. All

Instance Constructors

  1. new GibbsLDA (params: LDAModelParams, seed: Long, inferParams: GibbsInferParams, log: (String) ⇒ Unit)

Value Members

  1. def != (arg0: AnyRef): Boolean

    Attributes
    final
    Definition Classes
    AnyRef
  2. def != (arg0: Any): Boolean

    Attributes
    final
    Definition Classes
    Any
  3. def ## (): Int

    Attributes
    final
    Definition Classes
    AnyRef → Any
  4. def == (arg0: AnyRef): Boolean

    Attributes
    final
    Definition Classes
    AnyRef
  5. def == (arg0: Any): Boolean

    Attributes
    final
    Definition Classes
    Any
  6. def asInstanceOf [T0] : T0

    Attributes
    final
    Definition Classes
    Any
  7. var checkers : List[Function0[_]]

    Attributes
    protected
    Definition Classes
    RepCheck
  8. def checkrep (): Unit

    Assert invariants.

    Assert invariants.

    Attributes
    protected final
    Definition Classes
    RepCheck
  9. def clone (): AnyRef

    Attributes
    protected[lang]
    Definition Classes
    AnyRef
    Annotations
    @throws()
  10. def computeCrossEntropy (doc: LDADocumentParams): (Double, Int)

    Computes the total cross-entropy of the terms in the second half of the document based on an estimate of theta from the terms in the fisrt half of the doucment.

    Computes the total cross-entropy of the terms in the second half of the document based on an estimate of theta from the terms in the fisrt half of the doucment. Returns (sum crossEntropy, numTerms). This is used as the basis of computePerplexity.

    Definition Classes
    LDA
  11. def computeLogPW (doc: GibbsLDADocument): Double

    Computes the log probability for the current document.

    Computes the log probability for the current document. This measure treats the assignment to theta and the model counts as observed. Returns sum_i P(w_i | theta*, beta*). Beta maps from (topic,term) to probability.

    Definition Classes
    LDA
  12. def computePerplexity (docs: Traversable[LDADocumentParams]): Double

    Computes the average per-word perplexity of the given dataset.

    Computes the average per-word perplexity of the given dataset.

    Definition Classes
    LDA
  13. val countTopic : Array[Int]

    How many times each topic is seen overall.

    How many times each topic is seen overall.

    Definition Classes
    HardAssignmentModel
  14. val countTopicTerm : Array[Array[Int]]

    How many times each term is seen in each topic.

    How many times each term is seen in each topic.

    Definition Classes
    HardAssignmentModel
  15. def create (dp: LDADocumentParams): GibbsLDADocument

    Creates a document from the given document parameters.

    Creates a document from the given document parameters.

    Definition Classes
    GibbsLDATopicModel
  16. def eq (arg0: AnyRef): Boolean

    Attributes
    final
    Definition Classes
    AnyRef
  17. def equals (arg0: Any): Boolean

    Definition Classes
    AnyRef → Any
  18. def finalize (): Unit

    Attributes
    protected[lang]
    Definition Classes
    AnyRef
    Annotations
    @throws()
  19. def getClass (): java.lang.Class[_]

    Attributes
    final
    Definition Classes
    AnyRef → Any
  20. def getTopicTermDistribution (topic: String): Array[Double]

    Returns the distribution over terms for the given topic.

    Returns the distribution over terms for the given topic. The return value of this method is assumed to have already incorporated the corresponding getTermSmoothing to the appropriate extent.

    Attributes
    final
    Definition Classes
    ClosedTopicSet
  21. def getTopicTermDistribution (topic: Int): Array[Double]

    Returns the distribution over terms for the given topic.

    Returns the distribution over terms for the given topic. The return value of this method is assumed to have already incorporated the corresponding getTermSmoothing to the appropriate extent.

    Definition Classes
    ClosedTopicSet
  22. def hashCode (): Int

    Definition Classes
    AnyRef → Any
  23. def infer (doc: GibbsLDADocument): Array[Double]

    Does inference on the given document until convergence.

    Does inference on the given document until convergence.

    Definition Classes
    GibbsLDALDA
  24. def infer (doc: String): Array[Double]

    Does inference on the given document until convergence.

    Does inference on the given document until convergence.

    Definition Classes
    LDA
  25. def infer (doc: LDADocumentParams): Array[Double]

    Does inference on the given document until convergence.

    Does inference on the given document until convergence.

    Definition Classes
    LDA
  26. val inferParams : GibbsInferParams

  27. def inferSampler : InferSampler

    Gets a thread-local inference sampler.

  28. val inferSamplerTL : ThreadLocal[InferSampler]

    Attributes
    protected
  29. def isInstanceOf [T0] : Boolean

    Attributes
    final
    Definition Classes
    Any
  30. val learnSampler : LearnSampler

  31. val log : (String) ⇒ Unit

    Where log messages go.

    Where log messages go. Defaults to System.err.println.

    Definition Classes
    GibbsLDATopicModel
  32. def ne (arg0: AnyRef): Boolean

    Attributes
    final
    Definition Classes
    AnyRef
  33. def notify (): Unit

    Attributes
    final
    Definition Classes
    AnyRef
  34. def notifyAll (): Unit

    Attributes
    final
    Definition Classes
    AnyRef
  35. val numTerms : Int

    The number of terms in the model.

    The number of terms in the model.

    Definition Classes
    LDATopicModel
  36. val numTopics : Int

    The number of topics in the model.

    The number of topics in the model.

    Definition Classes
    LDAClosedTopicSet
  37. def pTopicTerm (topic: Int, term: Int): Double

    Returns the probability of the given term in the given topic.

    Returns the probability of the given term in the given topic.

    Attributes
    final
    Definition Classes
    HardAssignmentModelClosedTopicSet
  38. def pTopicTerm (topic: String, term: String): Double

    Returns the probability of the given term in the given topic.

    Returns the probability of the given term in the given topic.

    Definition Classes
    ClosedTopicSet
  39. val params : LDAModelParams

    The parameters used to create this model.

    The parameters used to create this model.

    Definition Classes
    GibbsLDALDATopicModel
  40. def registerCheck (check: Function0[_]): Unit

    Registers a function as a checker of invariants.

    Registers a function as a checker of invariants.

    Attributes
    protected
    Definition Classes
    RepCheck
  41. def reset (): Unit

    Resets to the default state.

    Resets to the default state.

    Definition Classes
    HardAssignmentModelStateful
  42. def sampleInfer (doc: GibbsLDADocument): Unit

  43. def sampleLearn (doc: GibbsLDADocument): Unit

  44. val seed : Long

  45. def state : HardAssignmentModelState

    Gets the current state of this object.

    Gets the current state of this object.

    Definition Classes
    HardAssignmentModelStateful
  46. def state_= (state: HardAssignmentModelState): Unit

    Sets the current state of this object.

    Sets the current state of this object.

    Definition Classes
    HardAssignmentModelStateful
  47. def summary : Iterator[String]

    Returns human-readable summary of the current topic model.

    Returns human-readable summary of the current topic model.

    Definition Classes
    HardAssignmentModel
  48. def synchronized [T0] (arg0: ⇒ T0): T0

    Attributes
    final
    Definition Classes
    AnyRef
  49. def termIndex : Option[Index[String]]

    The term index describing which terms are in the model.

    The term index describing which terms are in the model.

    Attributes
    final
    Definition Classes
    TopicModel
  50. def termIndex_= (index: Option[Index[String]]): Unit

    Attributes
    protected final
    Definition Classes
    TopicModel
  51. def termSmoothDenom : Double

    Attributes
    protected
    Definition Classes
    DirichletTermSmoothing
  52. def termSmoothing : Array[Double]

    Add-k prior counts for each term (eta in the model formulation).

    Add-k prior counts for each term (eta in the model formulation).

    Attributes
    final
    Definition Classes
    DirichletTermSmoothing
  53. def termSmoothing_= (smoothing: Array[Double]): Unit

    Attributes
    protected
    Definition Classes
    DirichletTermSmoothing
  54. def toString (): String

    Definition Classes
    AnyRef → Any
  55. def tokenize (document: String): Iterable[Int]

    Tokenizes the given input string using our stored tokenizer and term index, if available.

    Tokenizes the given input string using our stored tokenizer and term index, if available. Otherwise, throws an IllegalArgumentException.

    Attributes
    protected
    Definition Classes
    TopicModel
  56. def tokenizer : Option[Tokenizer]

    The tokenizer used to break input documents into terms.

    The tokenizer used to break input documents into terms.

    Attributes
    final
    Definition Classes
    TopicModel
  57. def tokenizer_= (tokenizer: Option[Tokenizer]): Unit

    Attributes
    protected final
    Definition Classes
    TopicModel
  58. var topicIndex : Option[Index[String]]

    The term index describing which terms are in the model.

    The term index describing which terms are in the model.

    Definition Classes
    ClosedTopicSet
  59. def topicName (topic: Int): String

    Gets the name for this topic.

    Gets the name for this topic.

    Definition Classes
    ClosedTopicSet
  60. def topicSmoothing : Array[Double]

    Prior counts for each topic (alpha in the model formulation).

    Prior counts for each topic (alpha in the model formulation).

    Attributes
    final
    Definition Classes
    DirichletTopicSmoothing
  61. def topicSmoothing_= (smoothing: Array[Double]): Unit

    Attributes
    protected
    Definition Classes
    DirichletTopicSmoothing
  62. def wait (): Unit

    Attributes
    final
    Definition Classes
    AnyRef
    Annotations
    @throws()
  63. def wait (arg0: Long, arg1: Int): Unit

    Attributes
    final
    Definition Classes
    AnyRef
    Annotations
    @throws()
  64. def wait (arg0: Long): Unit

    Attributes
    final
    Definition Classes
    AnyRef
    Annotations
    @throws()

Inherited from HardAssignmentModel[LDAModelParams, LDADocumentParams, GibbsLDADocument]

Inherited from LDA[HardAssignmentModelState, GibbsLDADocument, (String, Array[Int])]

Inherited from DirichletTopicSmoothing

Inherited from DirichletTermSmoothing

Inherited from ClosedTopicSet

Inherited from TopicModel[LDAModelParams, HardAssignmentModelState, LDADocumentParams, GibbsLDADocument, (String, Array[Int])]

Inherited from RepCheck

Inherited from Stateful[HardAssignmentModelState]

Inherited from AnyRef

Inherited from Any