Scalable knowledge-base construction with Microsoft Research
Large language models are increasingly being connected to external sources of knowledge, helping them remain up to date, provide more transparent answers, and avoid storing all information implicitly in model parameters. A major challenge is turning large, messy, unstructured text collections into reliable, structured knowledge bases. This requires identifying when different fragments of information refer to the same underlying entity, even when the evidence is incomplete, ambiguous, or arrives sequentially over time.
MARS is working with to develop scalable probabilistic algorithms for online model-based clustering, motivated by knowledge-base construction. The project uses Sequential Monte Carlo methods to represent uncertainty over possible clusterings, while introducing new mathematical structure that decomposes large clustering problems into approximately independent subproblems. This makes it possible to retain principled uncertainty quantification while reducing the memory and computational cost that usually prevents particle-based methods from scaling.