Markov logic network explained

A Markov logic network (MLN) is a probabilistic logic which applies the ideas of a Markov network to first-order logic, defining probability distributions on possible worlds on any given domain .

History

In 2002, Ben Taskar, Pieter Abbeel and Daphne Koller introduced relational Markov networks as templates to specify Markov networks abstractly and without reference to a specific domain.^[1] Work on Markov logic networks began in 2003 by Pedro Domingos and Matt Richardson.^{[2] [3]} Markov logic networks a popular formalism for statistical relational learning.^[4]

Syntax

A Markov logic network consists of a collection of formulas from first-order logic, to each of which is assigned a real number, the weight. The underlying idea is that an interpretation is more likely if it satisfies formulas with positive weights and less likely if it satisfies formulas with negative weights.

For instance, the following Markov logic network codifies how smokers are more likely to be friends with other smokers, and how stress encourages smoking:^[5]

Semantics

Together with a given domain, a Markov logic network defines a probability distribution on the set of all interpretations of its predicates on the given domain. The underlying idea is that an interpretation is more likely if it satisfies formulas with positive weights and less likely if it satisfies formulas with negative weights.

R that occurs in the Markov logic network and every n -tuple a _{1,} ... , a _{n} of domain elements, R(a _{1,} ... , a _{n)} is a grounding of R . An interpretation is given by allocating a Boolean truth value (true or false) to each grounding of an element. A true grounding of a formula \varphi in an interpretation with free variables x _{1,} ... , x _{n} is a variable assignment of x _{1,} ... , x _{n} that makes \varphi true in that interpretation.

Then the probability of any given interpretation is directly proportional to

\exp (\sum _{j} w _{j} n _{j)} , where w _{j} is the weight of the j -th sentence of the Markov logic network and n _{j} is the number of its true groundings.

This can also be seen as inducing a Markov network whose nodes are the groundings of the predicates occurring in the Markov logic network. The feature functions of this network are the groundings of the sentences occurring in the Markov logic network, with value

e ^{w} if the grounding is true and 1 otherwise (where again w is the weight of the formula).

Inference

The probability distributions induced by Markov logic networks can be queried for the probability of a particular event, given by an atomic formula (marginal inference), possibly conditioned by another atomic formula.^[6]

Marginal inference can be performed using standard Markov network inference techniques over the minimal subset of the relevant Markov network required for answering the query. Exact inference is known to be

1. P

-complete in the size of the domain.

In practice, the exact probability is often approximated.^[7] Techniques for approximate inference include Gibbs sampling, belief propagation, or approximation via pseudolikelihood.

The class of Markov logic networks which use only two variables in any formula allows for polynomial time exact inference by reduction to weighted model counting.^[8]

See also

• Markov random field
• Statistical relational learning
• Probabilistic logic network
• Probabilistic soft logic
• ProbLog

External links

• University of Washington Statistical Relational Learning group
• Alchemy 2.0: Markov logic networks in C++
• pracmln: Markov logic networks in Python
• ProbCog: Markov logic networks in Python and Java that can use its own inference engine or Alchemy's
• markov thebeast: Markov logic networks in Java
• RockIt: Markov logic networks in Java (with web interface/REST API)
• Tuffy: A Learning and Inference Engine with strong RDBMs-based optimization for scalability
• Felix: A successor to Tuffy, with prebuilt submodules to speed up common subtasks
• Factorie: Scala based probabilistic inference language, with prebuilt submodules for natural language processing etc
• Figaro: Scala based MLN language
• LoMRF: Logical Markov Random Fields, an open-source implementation of Markov Logic Networks in Scala

Notes and References

1. Taskar . Ben . Abbeel . Pieter . Koller . Daphne . 2002-08-01 . Discriminative probabilistic models for relational data . Proceedings of the Eighteenth conference on Uncertainty in artificial intelligence . UAI'02 . San Francisco, CA, USA . Morgan Kaufmann Publishers Inc. . 485–492 . 978-1-55860-897-9.
2. Book: Domingos, Pedro . The Master Algorithm: How machine learning is reshaping how we live. . 246-7 . 2015 .
3. Matthew . Richardson . Pedro . Domingos . Markov Logic Networks . Machine Learning . 62 . 1-2 . 107–136 . 2006 . 10.1007/s10994-006-5833-1 . PDF . free .
4. Building Practical Statistical Relational Learning Systems . UC Santa Cruz . 2023 . en . Eriq . Augustine.
5. Marra . Giuseppe . Dumančić . Sebastijan . Manhaeve . Robin . De Raedt . Luc . 2024-03-01 . From statistical relational to neurosymbolic artificial intelligence: A survey . Artificial Intelligence . 328 . 104062 . 10.1016/j.artint.2023.104062 . 0004-3702. 2108.11451 .
6. Sun . Zhengya . Zhao . Yangyang . Wei . Zhuoyu . Zhang . Wensheng . Wang . Jue . 2017 . Scalable learning and inference in Markov logic networks . International Journal of Approximate Reasoning . 82 . 39–55 . 10.1016/j.ijar.2016.12.003 . 0888-613X.
7. Venugopal . Deepak . 2017 . Advances in Inference Methods for Markov Logic Networks . IEEE Intelligent Informatics Bulletin . 18 . 2 . 13-19.
8. Kuzelka . Ondrej . 2021-03-29 . Weighted First-Order Model Counting in the Two-Variable Fragment With Counting Quantifiers . Journal of Artificial Intelligence Research . 70 . 1281–1307 . 10.1613/jair.1.12320 . 1076-9757. 2007.05619 .