Talk:Machine Intelligence Research Institute

NPOV for Pakaran

I've taken a lot of stuff out of the article that seemed to be basically just handwaving and self-promotion. This is what it read like when I found it:

"The Singularity Institute for Artificial Intelligence is a non-profit organization that seeks to create a benevolent artificial intelligence capable of improving its own design. To this end, they have developed the ideas of seed AI and Friendly AI and are currently coordinating efforts to physically implement them. The Singularity Institute was created in the belief that the creation of smarter-than-human, kinder-than-human minds represents a tremendous opportunity to accomplish good. Artificial intelligence was chosen because the Singularity Institute views the neurological modification of human beings as a more difficult and dangerous path to transhuman intelligence."

"The Singularity Institute observes that AI systems would run on hardware that conducts computations at billions or trillions of times the characteristic rate of human neurons, resulting in a corresponding speedup of thinking speed. Transhuman AIs would be capable of developing nanotechnology and using it to accomplish real world goals, including the further enhancement of their own intelligence and the consensual intelligence enhancement of human beings. Given enough intelligence and benevolence, a transhuman AI would be able to solve many age-old human problems on very short timescales."

As it stands, that isn't a bad article, it's just that it isn't really suitable for an encyclopedia. It presents some things as fact that are clearly opinion. It makes contentious statements, such as that it originated concept of "Seed AI" (astonishing for such a new organization--I read similar ideas in Von Neumann's book in the mid-seventies, and that had been written nearly thirty years before). The claim to be "coordinating efforts to physically implement" Seed AI and Friendly AI seem to rest on fundraising and writing a lot of papers about an extremely loosely defined programming language which seems to lack even an experimental implementation.

Wikipedia isn't for original research, it isn't for us to put up our pet ideas (however valid they may be). It's to catalog human knowledge from a neutral point of view. The article as it stood was in my opinion not so much an encyclopedia article as a promotional panegyric. --Minority Report 03:07, 23 Nov 2004 (UTC)

Ok, in the interest of admitting biases, I'm a financial donor to the SIAI. It's true that there have been holdups in beginning actual development, largely because there's a need to get all the theoretical underpinnings of Friendly AI done first.

That said, claiming that the SIAI is a "religion" rather than a group (which you may or may not agree with) is intrinsically PoV. --Pakaran (ark a pan) 03:55, 23 Nov 2004 (UTC)

I agree with most of your criticisms, Minority Report, and the article was not NPOV as it existed before. The statement that they are coordinating efforts to implement seed AI is quite valid, however. SIAI is developing a smaller, less ambitious AI program, although the primary objective of its research now is formalizing the theoretical framework for Friendly AI.

Also, using the phrase "quasi-religious" to describe an institution that claims to be entirely secular is highly misleading. SIAI has no affiliation with any religion.

I'm interested in your comments regarding von Neumann's work. I was not aware that von Neumann had speculated in this area. If you can find a source perhaps it should be mentioned at Seed AI. — Schaefer 05:02, 23 Nov 2004 (UTC)

I think my use of the term "quasi-religious" was an overstatement. I was trying to encapsulate the visionary aspect of this work, and the use of language which seems to owe more to religion than to engineering. I apologise if I also mischaracterized the Seed AI stuff; from looking around the site I saw a lot of hot air and little activity. I read a few books by Von Neumann in the late seventies, and the idea of having self-improving machines was very much his aim. I'm sorry I can't recall the specific book. I thought it might be The Computer and the Brain but a glance at the contents page on Amazon doesn't offer any clues. The idea was certainly in the air in the 1970s, long before Vinge's 1993 paper. --Minority Report 11:09, 23 Nov 2004 (UTC)

I've added basic information on the SIAI-CA and removed an erroneous middle initial. I also changed the first paragraph to reflect the fact that the SIAI actually does want to build software, rather than just talk about it, and to clarify that the 'Singularity' in the name refers to influencing the outcome of a technological singularity. --Michael Wilson

Merges

I have merged in information from the previously separate items on Emerson and Yudkowsky, which amounted to about a line of exposition and a few links. Those items now redirect to this item.

Yeah, I'd like that redirect to be removed. Actually, I'm removing it now. Eliezer Yudkowsky is wikified in many articles already. There is no reason to redirect an article about a person to their association's article. Biographical articles can be fleshed out over time and as of now it *looks* like we don't have an article on Yudkowsky when in fact we did. A line would have been a good start for smeone to write more. I'm making Eliezer Yudkowsky a bio-stub. --JoeHenzi 22:52, 21 Apr 2005 (UTC)

What is "reliably altruistic AI"?

Is it behavior that promotes the survival and flourishing of others at a cost to one's own? Wouldn't this require an AI with self-awareness and free will? But if an AI has free will, would it be moral to enslave it to serve the interests of others at the cost of it's own interests? Or is this merely a nod at Azimov's science-fiction "Three Laws of Robotics"? Those are close to a robotic version of a policeman's duties which may be seen as altruistic but may also be seen fulfilling a contract for which one is compensated. Or does the statement merely envision a non-self-aware AI with an analog of what ethologists call biological altruism? Whatever SIAI has in mind, I think the article should either make it explicit or drop the sentence since, as it stands, it is difficult or impossible to know what it means. Blanchette 04:43, 25 September 2006 (UTC)

The SIAI web site spends many pages addressing that question. Unfortunately I don't think it can be concisely added to this article, which is already fairly long; interested parties will just have to follow the references. Perhaps someone could expand the 'promotion and support' section of the 'Friendly Artificial Intelligence' article to detail the SIAI's definition of the term better. --Michael Wilson

Michael, thanks for the hint that what the author of the phrase "reliably altruistic AI" had in mind was the same thing as "Friendly AI". A search of the SIAI website reveals that the phrase "reliably altruistic AI" is not used there, nor is the term "reliably altruistic" nor is "altruistic AI". So "reliably altruistic AI" looks like an attempt to define Friendly AI that leads one away from rather than closer to understanding SIAI's ideas. I have replaced it with "Friendly AI" and the curious will then understand that further information is available through the previous link to "Friendly artificial intelligence". --Blanchette 07:06, 22 November 2006 (UTC)

Notibility

I just removed the notice about notability considering the institute has been written about in dozens of major publications. It's fairly obvious the notice doesn't belong. —Preceding unsigned comment added by 68.81.203.35 (talk) 15:21, 29 November 2008 (UTC)

Robotics attention needed

• Update
• Expand
• Check sources and insert refs
• Reassess once finished

Chaosdruid (talk) 08:30, 18 March 2011 (UTC)

Self-published papers?

Two of the several linked papers are even slightly peer-reviewed. Should these be linked? There is no evidence given that this work is noteworthy, either. If these extensive sections should be here, there needs to be evidence they're noteworthy and not effectively just an ad - David Gerard (talk) 11:53, 18 July 2015 (UTC)

When was SIAI->SI name change?

The SI->MIRI name change was January 2013. When was the SIAI->SI name change? I can't pin it down more closely than "some time between 2010 and 2012" - David Gerard (talk) 11:09, 10 April 2016 (UTC)

Neutrality?

User User:Zubin12 added a neutrality POV tag in this edit. However, the tag says "[r]elevant discussion may be found on the talk page," and the only discussion of neutrality issues on the talk page dates back to 2004. Per this guideline, the POV tag can be removed "[i]n the absence of any discussion." I'm going to remove the tag now, and if anyone feels the need to discuss the neutrality of the article, they can discuss it here first. --Gbear605 (talk) 00:29, 21 July 2018 (UTC)

Large amounts of Bias present

The article is most likely written by those supportive of the organization and it's mission, which is to be expected but that has caused a large amount of bias to appear in the article. Not only is much of the terminology used in the article confusing and not standardized but tons of tenous connections some of which I have removed.

The research section is incredibly confusing and next to impossible for a layman or even somebody not familiar with the specific sub-culture associated with the organization to follow, additional criticism or controversy about the organization remains limited. For this reason the article doesn't meat W:NPV standards Zubin12 (talk) 00:49, 21 July 2018 (UTC)

Thanks for adding your reasoning for the tag. I'm not entirely convinced it needs to be there, but I'm okay with leaving it for now. Gbear605 (talk) 01:07, 21 July 2018 (UTC)

I don't see how any of it is biased or confusing at all. Could you give some examples? K.Bog 15:15, 28 July 2018 (UTC)

It is a blatant advertisement, full of sources by the organization and other primary sources, and quotes that are not encyclopedic. This is not an encyclopedia article Jytdog (talk) 15:40, 28 July 2018 (UTC)

I think that's completely false. The primary sources here are being used for straightforward facts just like WP:PRIMARY says; it's okay to cite research to say what the research says. The presence of primary sources doesn't make something an advertisement. And the quotes seem perfectly encyclopedic to me. K.Bog 16:06, 28 July 2018 (UTC)

Hmm, that being said the research section does have some problems. So, I'll go ahead and fix it, and probably you will feel better about it afterwards.K.Bog 16:21, 28 July 2018 (UTC)

It is disgusting to see fancruft with shitty, bloggy sources on science topics. Video games, I understand more This is just gross. Jytdog (talk) 17:04, 28 July 2018 (UTC)

Please keep your emotions out of it. If you're not capable of evaluating this topic reasonably then move on to other things. Plus, I was in the middle of major edits, as I noted already. It's not good etiquette to change the article at the same time. I'm going to return it to the version I am writing, because I was working on it first, and then incorporate your changes if they are still relevant and suitable. K.Bog 17:29, 28 July 2018 (UTC)

"Disgust" is more an opinion, and one quite appropriate to blatant fan editing. This needs some serious non-fan review, and scouring of primary sources - David Gerard (talk) 17:40, 28 July 2018 (UTC)

But it's not fan editing, and primary sources are acceptable in the contexts used here. If you believe it requires third party review then flag it for actual third party review - you don't get to claim that you are unbiased if you have an axe to grind, whether it's negative or positive. K.Bog 17:44, 28 July 2018 (UTC)

@Jytdog you can finish if you want but you interrupted a major in-progress edit (this is the second time you did this to me, as I recall) and I'm going to revise it to my draft before looking at your changes.K.Bog 18:03, 28 July 2018 (UTC)

I wasn't aware that you were working over the page. That is what tags are for. Please communicate instead of edit warring. I will self revert. Jytdog (talk) 18:06, 28 July 2018 (UTC)

Thanks. I searched for the tag but couldn't remember what it was called. That's why I wrote it here on the talk page. K.Bog 18:11, 28 July 2018 (UTC)

Merge of changes

@User:Jytdog these are the significant differences between my version and your version:

• I kept the summary quote from the AI textbook because it is an easy to understand general overview of the research. One person here believed the article was too technical. I don't generally agree, but this quote is good insurance in case many people do find it too technical.
• I added Graves' article because it was published by a mainstream third party magazine and deals extensively with the subject matter.
• I have revised/streamlined the information about forecasting to read better.
• I have kept the AI Impacts info because it is referenced by reliable secondary sources.
• I kept brief references to all the papers that have been published in journals or workshops. Since they were published by a third party, they are notable enough for inclusion, and they follow WP:Primary, as they are being used to back up easily verifiable information about the subject ("X works on Y"). With these inclusions we have enough material to preserve all four research subsections.

The other things that you changed are things that I agree to change. I finished the article to my current satisfaction. Let me know if there is a problem with any of this or if the merge is complete K.Bog 19:40, 28 July 2018 (UTC)

There are still far too many primary or SPS refs. See below.

OK

• Hawking, Stephen; Tegmark, Max; Russell, Stuart; Wilczek, Frank (2014). "Transcending Complacency on Superintelligent Machines". The Huffington Post. Retrieved 11 October 2015.
• Basulto, Dominic (2015). "The very best ideas for preventing artificial intelligence from wrecking the planet". The Washington Post. Retrieved 11 October 2015.
• Ackerman, Elise (2008). "Annual A.I. conference to be held this Saturday in San Jose". San Jose Mercury News. Retrieved 11 October 2015.
• "Scientists Fear Day Computers Become Smarter Than Humans". Fox News Channel. Associated Press. 2007. Retrieved 12 October 2015.
• Abate, Tom (2006). "Smarter than thou?". San Francisco Chronicle. Retrieved 12 October 2015.
• Abate, Tom (2007). "Public meeting will re-examine future of artificial intelligence". San Francisco Chronicle. Retrieved 12 October 2015.
• Chen, Angela (2014). "More Rational Resolutions". The Wall Street Journal. Retrieved 5 March 2015.
• Tegmark, Max (2017). Life 3.0: Being Human in the Age of Artificial Intelligence. United States: Knopf. ISBN 978-1-101-94659-6. {{cite book}}: |access-date= requires |url= (help)
• Russell, Stuart; Norvig, Peter (2009). Artificial Intelligence: A Modern Approach. Prentice Hall. ISBN 978-0-13-604259-4.
• LaFrance, Adrienne (2015). "Building Robots With Better Morals Than Humans". The Atlantic. Retrieved 12 October 2015.
• Gallagher, Brian. "Scary AI Is More "Fantasia" Than "Terminator"". Nautilus. Retrieved 28 July 2018.
• Clarke, Richard A. (2017). Warnings: Finding Cassandras to Stop Catastrophes. HarperCollins Publishers. ISBN 0062488023.

bloggy but OKish

• Hsu, Jeremy (2015). "Making Sure AI's Rapid Rise Is No Surprise". Discover. Retrieved 12 October 2015.
• D'Orazio, Dante (2014). "Elon Musk says artificial intelligence is 'potentially more dangerous than nukes'". The Verge. Retrieved 5 October 2015.

churnalism

• De Looper, Christian (2015). "Research Suggests Human Brain Is 30 Times As Powerful As The Best Supercomputers". Tech Times. Retrieved 12 October 2015.

primary/SPS

(note sources by MIRI people are used as primary sources, where the content comments on them)

• "IRS Form 990" (PDF). Machine Intelligence Research Institute. 2013. Retrieved 12 October 2015.
• "Team". Machine Intelligence Research Institute. 2016. Retrieved 4 October 2016.
• Soares, Nate; Fallenstein, Benja (2015). "Aligning Superintelligence with Human Interests: A Technical Research Agenda". In Miller, James; Yampolskiy, Roman; Armstrong, Stuart; et al. (eds.). The Technological Singularity: Managing the Journey. Springer. {{cite book}}: External link in |chapterurl= (help); Unknown parameter |chapterurl= ignored (|chapter-url= suggested) (help)
• "Research Workshops". Machine Intelligence Research Institute. 2013. Retrieved 11 October 2015.
• GiveWell (2015). Potential risks from advanced artificial intelligence (Report). Retrieved 11 October 2015. {{cite report}}: Cite has empty unknown parameter: |coauthors= (help)
• "Singularity Institute Strategic Plan" (PDF). Machine Intelligence Research Institute. 2011. Retrieved 12 October 2015.
• "Singularity Summit: An Annual Conference on Science, Technology, and the Future". Machine Intelligence Research Institute. 2012. Retrieved 12 October 2015.
• Muehlhauser, Luke (2012). "July 2012 Newsletter". MIRI Blog. Retrieved 12 October 2015.
• Stiefel, Todd; Metskas, Amanda K. (22 May 2013). "Julia Galef" (podcast). The Humanist Hour. Episode 083. The Humanist. Retrieved 3 March 2015.
• Muehlhauser, Luke (2013). "We are now the "Machine Intelligence Research Institute" (MIRI)". MIRI Blog. Retrieved 12 October 2015.
• Bostrom, Nick (2014). Superintelligence: Paths, Dangers, Strategies (First edition. ed.). ISBN 0199678111.
• Muehlhauser, Luke (2015). "Musk and Gates on superintelligence and fast takeoff". Luke Muehlhauser Blog. Retrieved 12 October 2015.
• Future of Life Institute (2015). Research priorities for robust and beneficial artificial intelligence (PDF) (Report). Retrieved 4 October 2015. {{cite report}}: Cite has empty unknown parameter: |coauthors= (help)
• "2015 Awardees". Future of Life Institute. 2015. Retrieved 5 October 2015.
• Graves, Matthew (8 November 2017). "Why We Should Be Concerned About Artificial Superintelligence". Skeptic. The Skeptics Society. Retrieved 28 July 2018.
• Bostrom, Nick; Yudkowsky, Eliezer (2014). "The Ethics of Artificial Intelligence". In Frankish, Keith; Ramsey, William (eds.). The Cambridge Handbook of Artificial Intelligence. New York: Cambridge University Press. ISBN 978-0-521-87142-6. {{cite book}}: External link in |chapterurl= (help); Unknown parameter |chapterurl= ignored (|chapter-url= suggested) (help)
• Yudkowsky, Eliezer (2008). "Artificial Intelligence as a Positive and Negative Factor in Global Risk". In Bostrom, Nick; Ćirković, Milan (eds.). Global Catastrophic Risks. Oxford University Press. ISBN 978-0199606504. {{cite book}}: External link in |chapterurl= (help); Unknown parameter |chapterurl= ignored (|chapter-url= suggested) (help)
• LaVictoire, Patrick; Fallenstein, Benja; Yudkowsky, Eliezer; Bárász, Mihály; Christiano, Paul; Herreshoff, Marcello (2014). "Program Equilibrium in the Prisoner's Dilemma via Löb's Theorem". Multiagent Interaction without Prior Coordination: Papers from the AAAI-14 Workshop. AAAI Publications.
• Soares, Nate; Levinstein, Benjamin A. (2017). "Cheating Death in Damascus" (PDF). Formal Epistemology Workshop (FEW). Retrieved 28 July 2018.
• Soares, Nate; Fallenstein, Benja; Yudkowsky, Eliezer; Armstrong, Stuart (2015). "Corrigibility". AAAI Workshops: Workshops at the Twenty-Ninth AAAI Conference on Artificial Intelligence, Austin, TX, January 25–26, 2015. AAAI Publications.
• Taylor, Jessica (2016). "Quantilizers: A Safer Alternative to Maximizers for Limited Optimization". Workshops at the Thirtieth AAAI Conference on Artificial Intelligence.
• Hibbard, Bill. "Avoiding Unintended AI Behaviors" (PDF). Artificial General Intelligence: 5th International Conference, AGI 2012, Oxford, UK, December 8–11, 2012. Proceedings.
• Hibbard, Bill. "Decision Support for Safe AI Design" (PDF). Artificial General Intelligence: 5th International Conference, AGI 2012, Oxford, UK, December 8–11, 2012. Proceedings.</ref>
• Yudkowsky, Eliezer (2011). "Complex Value Systems in Friendly AI" (PDF). Artificial General Intelligence: 4th International Conference, AGI 2011, Mountain View, CA, USA, August 3–6, 2011. Berlin: Springer.

Jytdog (talk) 20:04, 28 July 2018 (UTC)

churnalism removed, I didn't notice it. Your list of primary/SPS is much too long because you are including a lot of separate organizations as well as authors. Bostrom is listed as an advisor on their website, not a member of the organization; if Russell is secondary then so is Bostrom. Givewell and FLI are separate entities. The Humanist Hour is a separate entity. They are not 'MIRI people.' And if an outside group writes or publishes on this group, it's not a primary source, it's a secondary source. e.g., FLI is only a primary source if they talk about themselves.

Also, some of those primary sources are being used in concert with secondary sources. If a fact is cited by both a relevant primary source and a secondary source saying the same thing, what compels you to remove the primary source? Of course, it doesn't really matter to me, so I've gone ahead and removed those, as well as some others from your list. The majority of sources are secondary, however there is no wiki policy that adjudicates on how much of an article can be primary sourced, as long as there are sufficient secondary sources. If an article can be made longer with appropriate use of primary sources, without being too long, then it's an improvement. Because more, accurate, information is simply a good thing.

Moreover, the section is no longer written like an advertisement. So neither tag is warranted.K.Bog 20:58, 28 July 2018 (UTC)

@Jytdog: There is only a single self-published source here, the FLI report, which satisfies WP:SPS. There are only about a dozen primary sources (i.e. papers written by people at MIRI) - less than half of the sources in the whole article, and all of them are published by third parties, and otherwise in accordance with WP:Primary. So the article mainly relies on secondary sources, therefore the primary source tag is unwarranted, see? As for advertisement - is there any specific wording in it that sounds like an advertisement? K.Bog 04:02, 25 August 2018 (UTC)

I have no words. I may have some later. Jytdog (talk) 04:19, 25 August 2018 (UTC)

arbitrary break

Content like this:

He argues that the intentions of the operators are too vague and contextual to be easily coded.

References

is still in the article. This is a primary source (a conference paper published on their own website and branded even) and the content is randomly grabbing some thing out of it. Not encyclopedic. This is what fans or people with a COI do (they edit the same way). There are a bunch of other conference papers like this as well and used in the same way. Conference papers are the bottom of the barrel for scientific publishing. There are still somewhat crappy blogs or e-zines like OZY and Nautlius.

A different kind of bad:

In early 2015, MIRI's research was cited in a research priorities document accompanying an open letter on AI that called for "expanded research aimed at ensuring that increasingly capable AI systems are robust and beneficial". Musk responded by funding a large AI safety grant program, with grant recipients including Bostrom, Russell, Bart Selman, Francesca Rossi, Thomas Dietterich, Manuela M. Veloso, and researchers at MIRI. MIRI expanded as part of a general wave of increased interest in safety among other researchers in the AI community.

References

In the first sentence

a) the first citation is completely wrong, which I have fixed.

b) The quotation doesn't appear in the cited piece (which is not the open letter itself, but rather says of itself "This article was drafted with input from the attendees of the 2015 conference The Future of AI: Opportunities and Challenges (see Acknowledgements), and was the basis for an open letter that has collected nearly 7000 signatures in support of the research priorities outlined here."

c) The cited document doesn't mention MIRI - this content saying "MIRI's research was cited in a research priorities document..." is pure commentary by who over wrote this; similar to the way the conference papers are used, discussed above. Again we don't do this.

In the second sentence:

a) The WaPo source doesn't mention the open letter. (I understand the goal here, but this is invalid way to do it)

b) The following people named as getting money, are not mentioned in the WaPo source: Russell, Selman, Rossi, Dietterich. However, Bostrom, Veloso, and Fallenstein at MIRI are mentioned. The WaPo piece mentions Heather Roff Perkins, Owain Evans, and Michael Webb. But this list has nothing to do with MIRI, so what is this even doing here?

The content is not even trying to summarize the source. This is editing driven by something other than the basic methods of scholarship we use here.

The third sentence:

The source here is the one that actually is telling the whole story of this paragraph. The reference lacks a page number (another issue of basic scholarship). It doesn't say that MIRI expanded per se; there is one sentence mentioning MIRI and it says "Major new Al­ safety donations enabled expanded research at our largest nonprofit sister organizations: the Machine Intelligence Research Institute in Berkeley, the Future of Humanity Institute in Oxford and the Cen­tre for the Study of Existential Risk in Cambridge (UK)."

I have fixed the paragraph here. Jytdog (talk) 15:00, 25 August 2018 (UTC)

arbitrary break with interspersed replies

Content like this:

He argues that the intentions of the operators are too vague and contextual to be easily coded.

References

is still in the article. This is a primary source (a conference paper published on their own website and branded even) and the content is randomly grabbing some thing out of it. Not encyclopedic. This is what fans or people with a COI do (they edit the same way). There are a bunch of other conference papers like this as well and used in the same way. Conference papers are the bottom of the barrel for scientific publishing. There are still somewhat crappy blogs or e-zines like OZY and Nautlius.

I've explained again and again that published primary sources are perfectly encyclopedic, and OZY and Nautilus are both published secondary sources. Computer science is different from other fields: most CS work is done in workshops and conferences rather than journals, and they are not considered "bottom of the barrel", so perhaps you aren't equipped to know what is reputable or not in the field of computer science. I don't know if you've actually looked at that citation either; the content in this article is roughly summarizing the thesis. If you want there to be *more* detail in this article, that's fine - feel free to add it yourself, but that's clearly not a reason to take away any details. So, I'm at a loss to see what the problem is. Perhaps you should familiarize yourself with the use of academic sources elsewhere on Wikipedia, because this is exactly how we write things all the time. K.Bog 21:38, 25 August 2018 (UTC)

A different kind of bad:

In early 2015, MIRI's research was cited in a research priorities document accompanying an open letter on AI that called for "expanded research aimed at ensuring that increasingly capable AI systems are robust and beneficial". Musk responded by funding a large AI safety grant program, with grant recipients including Bostrom, Russell, Bart Selman, Francesca Rossi, Thomas Dietterich, Manuela M. Veloso, and researchers at MIRI. MIRI expanded as part of a general wave of increased interest in safety among other researchers in the AI community.

References

In the first sentence

a) the first citation is completely wrong, which I have fixed.

Sure. Technical problem, I didn't write it, kudos to you for noticing.K.Bog 21:38, 25 August 2018 (UTC)

b) The quotation doesn't appear in the cited piece (which is not the open letter itself, but rather says of itself "This article was drafted with input from the attendees of the 2015 conference The Future of AI: Opportunities and Challenges (see Acknowledgements), and was the basis for an open letter that has collected nearly 7000 signatures in support of the research priorities outlined here."

Sure, that was probably a quotation from some other source that got lost in the perpetual churn and hacking. This is the kind of problem that articles have when people start revising them without paying any attention to the basic process of writing content.K.Bog 21:38, 25 August 2018 (UTC)

c) The cited document doesn't mention MIRI - this content saying "MIRI's research was cited in a research priorities document..." is pure commentary by who over wrote this; similar to the way the conference papers are used, discussed above. Again we don't do this.

No, that is a straightforward statement of fact, which is different from commentary. I presume that we do make straightforward statements of fact all the time.K.Bog 21:38, 25 August 2018 (UTC)

In the second sentence:

a) The WaPo source doesn't mention the open letter. (I understand the goal here, but this is invalid way to do it)

Okay. Then rewrite it to "Musk funded". K.Bog 21:38, 25 August 2018 (UTC)

b) The following people named as getting money, are not mentioned in the WaPo source: Russell, Selman, Rossi, Dietterich. However, Bostrom, Veloso, and Fallenstein at MIRI are mentioned. The WaPo piece mentions Heather Roff Perkins, Owain Evans, and Michael Webb. But this list has nothing to do with MIRI, so what is this even doing here?

I don't have WaPo access, so I don't know. Again I presume that the information was present across multiple sources, which got lost in one or more of your bouts.K.Bog 21:38, 25 August 2018 (UTC)

The content is not even trying to summarize the source. This is editing driven by something other than the basic methods of scholarship we use here.

You don't summarize the source, you summarize the part of the source that is relevant to the subject matter of the article. Maybe you should think more about this sort of thing before throwing accusations around. K.Bog 21:40, 25 August 2018 (UTC)

The third sentence:

The source here is the one that actually is telling the whole story of this paragraph. The reference lacks a page number (another issue of basic scholarship).

"Basic scholarship"! My ebook lacks page numbers so I do not know which page it's on, but somehow you assume that I am bad at basic scholarship? That's rather arrogant on your part. Please do better in the future. K.Bog 21:38, 25 August 2018 (UTC)

It doesn't say that MIRI expanded per se; there is one sentence mentioning MIRI and it says "Major new Al­ safety donations enabled expanded research at our largest nonprofit sister organizations: the Machine Intelligence Research Institute in Berkeley, the Future of Humanity Institute in Oxford and the Cen­tre for the Study of Existential Risk in Cambridge (UK)."

Expansion of research at a research group = expansion. It would be idiotic to bicker over this level of semantics.K.Bog 21:38, 25 August 2018 (UTC)

I have fixed the paragraph here. Jytdog (talk) 15:00, 25 August 2018 (UTC)

Mention of Nick Bostrom

Kbog, why are you edit-warring a tangential mention of Nick Bostrom in? If he's MIRI it's self-sourced puffery, and if he's not then it's tangential. Having lots of blue numbers after it - one of which is a Bill Gates interview on YouTube with no mention of MIRI - doesn't make it look cooler or something - David Gerard (talk) 08:17, 25 August 2018 (UTC)

That mention was in both my and Jytdog's versions of the article, clearly I am not trying to edit war anything *into* the article, this is typical bold-revert-discuss, exactly how things are supposed to work. The relevance is that it is background for the expansion of interest and funding for the organization. E.g. in the World War II article, the "Background" section has a mention of World War I, and that is not tangential. You are right that Gates is not relevant, I took him out of it. Puffery is non-neutral language, like "esteemed", "highly regarded", etc - whereas this article uses plain factual language. Anyway the current wording should make it more clear - I can now see how the previous wording might have made it appear out of place and gratuitous. K.Bog 08:28, 25 August 2018 (UTC)

User:Kbog The current wording is a still a bit puffy,and the article has the endemic problem of simply quoting the opinion of others rather being a real encyclopedic entry. There are far too many "Believes" and "Argued" in the article for it to qualify as a proper encyclopedia entry. — Preceding unsigned comment added by Zubin12 (talk • contribs) 00:58, 26 August 2018 (UTC)

"Believes" and "argues" are not puffy, they are neutral, factual representations of people's positions: they neither support nor condemn any ideas, they state an uncontroversial fact about what people are saying. If you want an alternative, we can remove them with pure statements of the material, e.g., replace "Muelhauser and Bostrom argue that hard-coded moral values would eventually be seen as obsolete" with "hard-coded moral values would eventually be seen as obsolete", and so on. But proper attribution of arguments seems like a more encyclopedic way of doing things. K.Bog 01:26, 26 August 2018 (UTC)

A lot of those people paraphrased or quotes in the article have only a tangential connection to the MIRI institute. The research section needs a total re-write to include the only research that was undertaken by the institute or was closely connected with it. Only Notable or relevant arguments/opinions should be included in the encyclopedia.Zubin12 (talk) 01:44, 26 August 2018 (UTC)

I don't know of any Wikipedia policy against statements that have a tangential connection to the subject of the article. All the research in the research section is about research done by people at the institute or closely connected with it. All the arguments and opinions here are published by third parties, so they are notable enough for inclusion.K.Bog 01:48, 26 August 2018 (UTC)

In your last revert, you said, "The fact that Musk Endorsed the book is irrelevant, your argument that he needed to be mentioned because he gave the MIRI a grant is superflous". I don't know what you mean by this: "superfluous" means "redundant"; if my argument is redundant then I am right anyway. The "history" section is a story about relevant things that have happened in the past, and part of this story is Musk's involvement. Anyway, you cannot remove material without building a talk page consensus first. Merely saying "let's take this to the talk page" doesn't give you a right to continue edit warring. The material stands until there is a decision to change it. K.Bog 01:53, 26 August 2018 (UTC)

As a third party editor, I agree with Kbog that the reversion done in https://en.wikipedia.org/w/index.php?title=Machine_Intelligence_Research_Institute&diff=next&oldid=856553209 should be essentially left as is until talk page consensus is reached. Gbear605 (talk) 01:57, 26 August 2018 (UTC)

OK, I have changed it to what I presume is an agreeable rewrite - the Musk endorsement is now placed in the final paragraph, next to his actions, so that no one mistakes it for being irrelevant. K.Bog 02:02, 26 August 2018 (UTC)

The article is about the MIRI, not every article that has cited or been influenced by the MIRI.

1.) Mentioning that Nick Bostrom's book has been endorsed by Elon Musk isn't related to the MIRI directly, only through tenous second-order effects. It's enough to say that the book helped spark a public discussion that drew attention to the organization. The fact that Musk Endorsed it has zero Direct connection to the subject of the article. The fact that the MIRI got a grant due to a conference organized by Musk is mentioned separately.

2.) The fact that this followed-up by an Huffington post article by 2 separate respected and famous people, just make it puffery by name-dropping famous people who hae mentioned the organization build up it's credibility.

3.) The research section's language is convoluted and needs to be simplified to a more understandable and general level. The sourcing and citations itself are fine, but the level and non-standard terminiolgy used makes it seem like fan-crutf.(Additionally, just like I behaved inapportiably by edit-warring, the Tag Should be re-added as they should not be removed until after talk-page discussion achieves consensus). Zubin12 (talk) 02:08, 26 August 2018 (UTC)

I agree that the tags should be re-added until consensus is reached. I'm adding them back now.Gbear605 (talk) 02:11, 26 August 2018 (UTC)

Per Wiki Policy, a maintenance template should be removed immediately if there is a consensus that its initial placement was in error (note: consensus doesn't necessarily mean "everyone agrees"). We can positively see from the article that the block quotes are a small part of it, and that the language imparts real information in a neutral factual manner. Zubin still hasn't actually attempted to justify either tag, he has merely stated that it is too technical and has some irrelevant information, which *he feels* are promotional, even though there is no basis for this inference in the Wikipedia manual of style or anywhere else. If he will still refrain from forming a proper argument then it's an uncontroversial case for reversion. K.Bog 02:42, 26 August 2018 (UTC)

Hmm, I was under the misapprehension that the tags were there prior to Zubin12's editing of this page. However, Zubin added them with https://en.wikipedia.org/w/index.php?title=Machine_Intelligence_Research_Institute&diff=prev&oldid=851150526. With that in mind, I would support going to a single POV tag (since the neutrality is clearly under dispute) until either a third party editor - who actually has a view about it beyond stopping the edit war - reviews this page, or the two of you manage to reach an agreement. Gbear605 (talk) 02:52, 26 August 2018 (UTC)

The previous tags for POV sorts of things were for issues that were raised and resolved above. The current version of the article has all of Jytdog's edits, and the only dispute so far has come from Zubin. K.Bog 03:03, 26 August 2018 (UTC)

1. It has a rather clear second-order connection, and that is okay. For instance, in the World War II article, it mentions the civil war between the KMT and the communists, because that has a second-order connection to the Sino-Japanese part of the conflict. That's included even though it is not technically about World War II. As for this article, the fact that Musk endorsed Bostrom's arguments has a rather clear second-order connection to the fact that he paid people to do research on them.

2. We are going out of our way to stack the article with reliable, notable sources, and that means articles in famous media outlets that quote respected and famous people. What bizarre theatrics are these, where the very defenses that exist to ward off deletionist gadflies are used as a pretext for their further persistence. It seems that literally everything, to you, is a reason to be combative: either it's not notable enough so it must be removed, or it's too notable so it must be puffery; either it's irrelevant, or it's so relevant that it's superfluous; and so on. Instead of doing this, you should stick to a straight and consistent application of Wikipedia's policies.

3. It's quite easy to understand, especially compared to other technical issues (see e.g. [1], which doesn't seem like "fan-cruft" to me), so I don't see the problem. It is just my judgement as a native English speaker that they are as understandable as possible without losing detail or encyclopedic tone; if you disagree, then propose a rewrite. AFAIK, no one else has found that this article is hard to understand. And that's not how tags work: they are not missiles that you can fire-and-forget, you don't get to tag-bomb a page just because you personally don't agree with it, they are subservient to editors' consensus rather than being above it. If other editors think the page is fine, then you need to show what the problem is. K.Bog 02:34, 26 August 2018 (UTC)

1. It's not a rather clear second-order connection, but a tenuous third-order connection. The chain of logic between the grants and the publication of the book takes 3 parts. There is a clear interest in explaining the background of a conflict that laid the ground-work for world war 2 is asia, that simply isn't present in mentioning the endorsement of an unaffiliated book. For example, if Musk's endorsements of the book caused the MIRI to be founded or reforms then it would be notable.

This is pointless bickering, please phrase your objection in terms of WikiPolicy. For instance, if there is a WikiPolicy against third-order connections, then go by that. Otherwise there is no point arguing about the number of steps in the chain of logic. You are begging the question by saying that there is no clear interest here: either introduce a significant argument, or concede the point. K.Bog 03:15, 26 August 2018 (UTC)

WP:Synth prohibits drawing conclusion from two sources that aren't explicitly stated, unless you find a source explicitly saying that without musk's endorsement of the Book the grant wouldn't have gond through or a public discussion wouldn't have started it would be prohibited by that.Zubin12 (talk) 03:29, 26 August 2018 (UTC)

Nowhere in the article is it stated that Musk wouldn't have made the grant without the book, so this complaint is spurious. If you think there is an OR problem then identify a statement in the article that constitutes OR. What the article does is it states Musk's endorsement of Bostrom's arguments right before talking about his actions to fund work that is closely related to Bostom's book. The relation between these things is banal, not synthetic. K.Bog 03:42, 26 August 2018 (UTC)

Can you see the catch-22 in what you are saying?. Either his endorsement of the book has no relation to MIRI Grant in which case it has no relation to the subject article and should be removed, or else it does share a relation unmentioned in the sources in which case mentioning it is a violation of WP:Synth.Zubin12 (talk) 03:57, 26 August 2018 (UTC)

It has a relation which is banal and non-synthetic, viz. that they are both about Musk's concern for superintelligence risk. There is a difference between actual synthesis, and taking two things that are obviously relevant to each other and writing them next to each other. K.Bog 04:07, 26 August 2018 (UTC)

, They might be related to each other but one of them isn't related to the topic of the article. Let's say the topic of an Article is X, Musk's Grant to the MIRI is Y, and his endorsment of the book is Z. X is Connected to Y, and Y is Connected to Z but Z is not connected to X and therfore shouldn't be included unless their is a source connecting it directly to the MIRI.Zubin12 (talk) 04:28, 26 August 2018 (UTC)

No such rules exist on Wikipedia, nor should they. K.Bog 05:07, 26 August 2018 (UTC)

2. The history page inflates a passing a mention, into a "Citation" and block-quotes an article with a single mention of the organization. I don't see any reason for including it in the history section of the article as it doesn't seem to have had any effect on the organization whatsoever. A source must be both Notable and Relevant to merit inclusion, many sources used in the article fail these criteria.

You're right that it has a single mention of the organization, but again I don't see the problem. That single mention is what is quoted here, with one previous statement for context. The Tegmark reference in this article makes clear the relation with MIRI's work, so perhaps you should read that first. You haven't pointed out how even a single source here is non-notable or not relevant, except perhaps for some of the other things I'm answering right here. K.Bog 03:16, 26 August 2018 (UTC)

WP:Due, a single reference as an example in an article demands at best a passing mention, rather than a full on-block quote. Unless the Huffington post article led to changes or an event at the MIRI then it shouldn't be block-quoted as it's placing an undue focus on a minor event.Zubin12 (talk) 03:29, 26 August 2018 (UTC)

The article talks extensively about the subject of AI safety, so it is not a passing mention. MIRI is written as one of the few examples of the primary issue raised by the article. Just because it only states the name once doesn't mean that's the only relevant bit. If it has too much weight relative to other sources, then add more details from those other sources. K.Bog 03:42, 26 August 2018 (UTC)

This article is about the MIRI not AI Saftey. The Huffington Post Article mentioned the MIRI institue as one example of an organization working on the issue, point out where else in the article is idea formulated by the MIRI are mentioned. Zubin12 (talk) 03:57, 26 August 2018 (UTC)

Yes, and MIRI works on AI Safety. Tegmark's book explicitly points out that the HuffPo article was about the AI safety issue that Yudkowsky helped raise, so I do not need to explain that relation myself. K.Bog 04:07, 26 August 2018 (UTC)

. That's Textbook WP:Synth, you are making a connection based on 2 sources that aren't mentioned in them. Zubin12 (talk) 04:28, 26 August 2018 (UTC)

I am not making a connection, I am writing them next to one another. There is some basic level of connections that we always make, e.g. if Finland's population was 10 million in one century, and 20 million the next, we can state those two facts next to one another without having a source that makes a connection. That's not what WP:Synth is about. K.Bog 05:04, 26 August 2018 (UTC)

3. The Kernel Article uses industry standard terminolgy, it might not be understandable for a layman but a person with a basic understanding of computer science will be able to grasp what it is talking about. By comparison this article is written in a way to puff-up meager insight, take a look at this line "Their work includes formalizing cooperation in the prisoner's dilemma between "superrational" software agents[20]and defining an alternative to causal decision theory and evidential decision theory.[21]", an almost incomprehensible line for anybody not familiar with the MIRI. Stylistic problems abound, Pronounces are used inconstantly and sentence structures are constantly repeated. Zubin12 (talk) 03:01, 26 August 2018 (UTC)

All those words are standard in decision theory, as can be discovered by merely Googling them, and anyone with a basic understanding of decision theory will grasp what it is talking about. I don't see any stylistic problems or "inconstant [sic]" use of "Pronounces [sic]", and I don't see any sentence structure that's repeated too much. To be blunt, you don't seem to be a native English speaker, your own bio states that you are from Singapore and are on Wikipedia in order to improve your grammar and spelling, and your writing here has many errors. While it's admirable of you to make such an effort and learn, you should understand our skepticism regarding your perceptions of what is or isn't good English style. K.Bog 03:15, 26 August 2018 (UTC)

I am a native speaker of English, my grammar and spelling is horrid so i'm seeking to improve that by writing more. It's kinda chauvinistic to think that only second-language speaker would ever want to improve their language skills. The phrase "MIRI research" is used 3 times in a single paragraph, and the sentence structures are overcomplicated.For your information, English is the second most common first language in Singapore along with our Lingua Franca so don't make assumptions based on Partial Information.Zubin12 (talk) 03:29, 26 August 2018 (UTC)

Whether it's a matter of first language or just unfamiliarity with decision theory, it's evident that you do not know the language that you are talking about here: with blue links, we can write "their work includes formalizing cooperation in the prisoner's dilemma between "superrational" software agents[20]and defining an alternative to causal decision theory and evidential decision theory," and in these articles we can see that these terms are used in all sorts of places besides MIRI. As for reusing the same phrase, or having "overcomplicated" sentence structures: again these appear to be total non-issues; I don't see anything like that to any problematic extent. There mere repeat of a phrase does not bother me. But if you want to fix these without removing any detail then go ahead. For instance, you can replace "MIRI research" with "work at MIRI" or other similar phrases, and so on. Either way, it has nothing to do with POV, promotionalism, or any such maintenance tag. By the way, you're wrong about the Kernel article as well: lots of computer science students have no idea what a positive definite matrix is, what a Gram matrix is, and other bits of it. K.Bog 03:42, 26 August 2018 (UTC)

I gave one example earlier of repetion of a single phrase in a paragraph, anyway this was another issue with the article. My other 2 points deal with the NPOV problem present in the article. Other people have previously commented on the stylistic problem in the article, which has remained unresolved even if the discussion have gone dormant.Anyway WP:Otherstuff exists makes that line of argument spurious.Zubin12 (talk) 03:57, 26 August 2018 (UTC)

The article has been substantially revised since most of the comments here. I don't see any recent comments on style aside from Jytdog's, but it seems that all his stylistic changes have been included. So it seems that it's just you. This is not 'otherstuff', this is an appeal to common sense good practice. K.Bog 04:07, 26 August 2018 (UTC)

@David Gerad: seems to agree that my changes should be kept. It seems like the discussion isn't going anywhere and it's unlikley we will be able to convince each other or agree on a compromise in the current discussion. I"m going to take the liberty of pinging other editors of the artricle, for their view on our dispute. What are your thoughts? @Gbear605, David Gerard, and Jytdog:, Please Ping any other editors you feel might help resolve this dispute. Zubin12 (talk) 04:28, 26 August 2018 (UTC)

Zubin12, I agree that it looks like you won't be able to come to a compromise. I'm not sure what the correct solution is, but I definitely agree that others need to be called in. Gbear605 (talk) 04:30, 26 August 2018 (UTC)

Gbear605, Kbog Should a formal RFC be initated ? Zubin12 (talk) 04:35, 26 August 2018 (UTC)

Yes, I have made one. K.Bog 05:35, 26 August 2018 (UTC)

yes there is a boatload of completely offtopic stuff here, inappropriately sourced at that. Including the bit you are discussing. Jytdog (talk) 04:29, 26 August 2018 (UTC)

Primary sources

As has been stated repeatedly above, the primary sources in this article meet the standards in WP:PSTS, and are a minority of the sources being used in the article. The only person here who has raised a problem with the use of primary sources here is Jytdog. Jytdog, you are perhaps unfamiliar with how articles on academic subjects on Wikipedia are normally written: published papers are standard, and comprise the bulk of reliable information on a topic. See, to pick an arbitrary example, Kantian ethics, which has been labeled a "good article," despite citing Kant himself numerous times, and citing other people for their own views. It's inappropriate to slap a maintenance template on the page when these points have been made repeatedly without being answered. K.Bog 04:24, 26 August 2018 (UTC)

Stated repeatedly by you. Your use of primary sources is not appropriate, as me and the two other people are trying to explain to you. You are not listening. Jytdog (talk) 04:27, 26 August 2018 (UTC)

I've clearly stated how it is appropriate, and you haven't responded. Nowhere have you or anyone else pointed out a real problem with the primary sources that are currently in the article. Zubin's comments are about a secondary source, the HuffPo article. No one else has stated any problems with the recent version of the article since the sourcing was revised. The only thing you have said on this matter is that the papers are from conferences, but as I have explained already, it is normal for computer science papers to be published in conferences. Now you're resorting to flat denial. I'm sorry, but if you don't give a sound argument, then you can't assume that people are "not listening" when they still disagree. K.Bog 04:41, 26 August 2018 (UTC)

Kbog, so far this talk page discussion is you vs. everyone else - David Gerard (talk) 10:32, 26 August 2018 (UTC)

The Musk bit is puffery, and the cites regarding Future of Life Institute literally didn't mention Future of Life! Removed - David Gerard (talk) 13:08, 26 August 2018 (UTC)

and the book cite ... turned out to be a claim from Tegmark in his role at FLI that their programme had encouraged more grants! I've changed the mention to note that this is a first-party assertion - not any sort of third-party-verified factual claim. It's also literally the only mention of MIRI in the book. I've left it for now, but really, without something that's at least third-party RS as a source, this claim is unsupported and shouldn't be here - David Gerard (talk) 13:16, 26 August 2018 (UTC)

Actually, it doesn't even support the claim - the quote (which, as I note, is literally the only mention of MIRI in Life 3.0) is "Major new AI-safety donations enabled expanded research at our largest nonprofit sister organizations: the Machine Intelligence Research Institute in Berkeley, the Future of Humanity Institute in Oxford and the Centre for the Study of Existential Risk in Cambridge (UK)." That does not support the claim made in the text - this is a bogus citation. Removing the claim - David Gerard (talk) 13:41, 26 August 2018 (UTC)

The other claim cited to Life 3.0 is:

In fall 2014, Nick Bostrom's book Superintelligence: Paths, Dangers, Strategies[9] helped spark public discussion about the work of researchers such as Yudkowsky on the risk of unsafe artificial general intelligence.

As well as the sole MIRI mention not supporting this claim, the book mentions Yudkowsky precisely once, in a footnote:

Eliezer Yudkowsky has discussed aligning the goals of friendly AI not with our present goals, but with our coherent extrapolated volition (CEV). Loosely speaking this is defined as what an idealized version of us would want if we knew more, thought faster and were more the people we wished we were. Yudkowsky began criticizing CEV shortly after publishing it in 2004 (http://intelligence.org/files/CEV.pdf), both for being hard to implement and because it’s unclear whether it would converge to anything well-defined.

This sentence fails verification, and the cited reference is irrelevant to MIRI and shouldn't be in this article. I've marked it as failing verification for now, but it really needs a third-party RS citation that verifiably checks out - David Gerard (talk) 13:48, 26 August 2018 (UTC)

Request for comment on NPOV and sourcing

The points of dispute are: (a) whether to remove the statement that Musk "had previously endorsed Bostrom's arguments,[11][12]" (b) whether to preserve the NPOV maintenance tag, (c) whether to preserve the primary source maintenance tag, and (d) whether the language is too technical. K.Bog 05:13, 26 August 2018 (UTC)

• Your cites are bad and fail to check out, or turn out to be first-person assertions rather than the third-party verified claims they turn out to be, as I just noted above. This does not bode well. The article may need to be reconstructed from the ground up purely from verifiable and verified third-party RSes - not primary sources, not sources linked to MIRI, not sources making claims about themselves. Only then should we see about seasoning it with permissible primary sourcing and so on - David Gerard (talk) 13:19, 26 August 2018 (UTC)

Notes

• This is not an RfC about policy; I have removed that tag from the RfC because per WP:RFCST that The "Wikipedia policies and guidelines" category is for discussing changes to the policies and guidelines themselves, not for discussing how to apply them to a specific case. The same applies to "style", "WikiProject", and the other non-article categories. Jytdog (talk) 14:01, 26 August 2018 (UTC)