Self-XSS (self cross-site scripting) is a attack used to gain control of victims' web accounts. In a Self-XSS attack, the victim of the attack runs in their own web browser, thus exposing personal information to the attacker, a kind of vulnerability.
Self-XSS operates by tricking users also into copying and pasting malicious content into their browsers' web developer console. Usually, the attacker posts a message that says by copying and running certain code, the user will be able to receive virtual rewards or hack a website. In fact, the code allows the attacker to hijack the victim's account.
In the past, a very similar attack took place, in which users were tricked into pasting malicious JavaScript into their address bar. When browser vendors stopped this by preventing easily running JavaScript from the address bar, attackers started using Self-XSS in its current form. Web browser vendors and web sites have taken steps to mitigate this attack. Firefox and Google Chrome have both begun implementing safeguards to warn users about Self-XSS attacks. Facebook and others now display a warning message when users open the web developer console, and they link to pages explaining the attack in detail.
The "self" part of the name comes from the fact that the user is attacking themselves. The "XSS" part of the name comes from the abbreviation for cross-site scripting, because both attacks result in malicious code running on a legitimate site. However, the attacks do not have much else in common, because XSS is an attack against the website itself (which users cannot protect themselves against but can be fixed by the site operator making their site more secure), whereas Self-XSS is a social engineering attack against the user (against which savvy users can protect themselves but the site operator cannot prevent).