Talk:Literate programming

On April 26, 2008, Literate programming was linked from Slashdot, a high-traffic website. (Traffic)

All prior and subsequent edits to the article are noted in its revision history.

Suitability of the example

What the hell is the language used in the example? It appears that the stupid language in the blue boxes is the literate program and thus that literate programming is that stupid language. The example should use a mainstream language like C or Java to make it apparent that literate programming is not itself a programming language, per se. It should also use the features which distinguish literate programming's code-in-documentation approach from the documentation-in-code systems like JavaDoc. -Tim can't be arsed to sign in. —Preceding unsigned comment added by 82.68.187.214 (talk) 19:41, 26 April 2008 (UTC)[reply]

The example given is not literate programming (follow the link at the bottom of the page to see what literate programming is about). This article should be wound back, or the example section replaced with a better example (e.g. a java class with javadoc-style comments). See also Talk:Interpreter (computer software) -- AndrewKepert 04:56, 25 Aug 2003 (UTC)

A literate program is a descriptive document for people containing program text marked in such a way that a literate programming system can tell the difference between the program text and the rest of the document for processing purposes. Take a look at the article/program contained in the adventure.pdf file on the literate programming site. In principle there is no difference between that article and this one apart from the method used to mark the program text so it puzzles me that you do not consider this article to be an example of literate programming. Perhaps you can give your reasons in a little more detail.

As far as I can see, the entire Literate programming article constitutes a simplified literate programming example, usable by people for tutorial purposes, or by its "literate programming system", ie the example interpreter on the interpreter page, as stated. The facts that it does not extract the program code from the article to make a separate program, that it is not usable as much more than an example, and that it is not a CWEB system nor any of the other more functional literate programming systems, are neither here nor there. It is intended to be a simple example of the essence of a literate program which avoids the clutter introduced to real systems by the need for functions which are not directly related to literate programming or which are needed for realistically sized programs.

Furthermore this type of simple literate programming system is not original to me. I first saw it described as an Awk implementation in one of Jon Bentley's excellent Programming Pearls books in the early 1990s.

Admittedly the article does not tell the whole story about literate programming but that is not unusual for the Wikipedia. The answer to that is either to expand the article or, as you say, to follow the link at the bottom of the page. -- Derek Ross

The example is essentially "code with comments", which is not what literate programming is. There is no description of the algorithms and intent in the design of the program, nor of decisions made in implementation. The point of literate programming is that the code is secondary and follows from this description. To a purist, the code need not even be read except to check that the implementation does what it the decription says. The example "adventure.pdf" has sections describing algorithms and storage (e.g. Section 19, sections 71+) which follow this structure, and other sections (23-70 -- essentially data dumps) where this is relaxed as the code becomes more self-documenting (particularly in light of some well-chosen macros and function names). The reason for my earlier brief comment was that the example is a poor example of literate programming since it is indistinguishable from non-literate programming. If there is a large grey area (which in this case includes a lot of javadoc stuff too) then examples given should strive to highlight what separates literate from other programming. To illustrate this with your code, I could suggest some modifications

First let's find out the user's name
- In the interests of putting the user at ease, the program will simulate personal interest in the user by asking for their name, accepting the input and generating a greeting based on the input text.
Now let's work out the area of a circle. The user will be asked for the radius first.
- Continuing the "query-response" mode of operation, prompt the user for the radius of a circle, which is then used to calculate the area of a circle using the standard formula for the area of a circle: A = πr². Due to syntax limitations, this is done by multiplying the input value by itself, then by π. This calculated value is returned to the user. Note: the value of π used is an approximation that is sufficiently accurate for our purposes.

However, I still don't think such suggestions shift the example sufficiently far from the grey area to make it a good example. Others' opinions may differ. Cheers, AndrewKepert 22:42, 25 Aug 2003 (UTC)

Okay, that makes it clear. The example is poor and needs improving. Well, I can live with that. Once I contribute something to Wikipedia, it belongs to Wikipedia, not to me, and I expect it to be improved mercilessly. Personally, I'd be quite happy to add your text to the article and will do so since it's manifestly better than the comment-like text which I used. On the other hand, you seem to think that the example needs total replacement. I don't think that that is really necessary but if you feel strongly about it, it's fine by me. My only proviso is that the example isn't too large and is based on a WEB like system rather than JavaDocs. The reason that I say this is that JavaDocs appears to me to be a system for processing programs containing document generation code rather than a system for processing documents containing program text. It's a subtle point but I think that it's important. To me the latter is what Knuth meant by "literate programming". -- Derek Ross

IMHO a good example here need not be accessible as a piece of code, as the code (or fragment thereof) is embedded in the documentation. Consequently, there is nothing wrong with pulling a fragment (© allowing) from a well-written piece of real-world literate programming as an example. Having the algorithm/etc described and then the implementation would be a valuable illustration, even if the language of implementation is somewhat techincal. One possible source of examples is the LaTeX project. Its source files are in the form of self-documenting .dtx files. These can be typeset (via LaTeX of course) on their own to produce documentation (in the form of paragraphs with embedded snippets of code), or processed using docstrip to produce the style, class, etc files. I will have a scrounge through tetex to see if I can find a good small example. The reason why I (regrettably) orginally suggested javadoc is that someone might be able to dig out a good example that is small and illustrative of the qualities of LP. -- AndrewKepert 08:37, 27 Aug 2003 (UTC)

PS btw I will fix the formula from "pr2" to πr² -- AK

Sounds good, Andrew. Please go ahead. -- Derek Ross

Code Blocks

Another problem with the example is that it does not show code block rearrangement, though this is claimed as crucial for the effectiveness of literate programming before. -- Ar 13:16, 2004 Dec 5 (UTC)

Well I do have an enhanced version of the current literate programming interpreter with a preprocessing stage which can handle code block re-arrangement too but I thought it a little too complex for the interpreter article. I suppose that I could add it to WikiSource and put links to it from this article. -- Derek Ross | Talk 15:50, 2004 Dec 6 (UTC)

Mention of Haskell

Someone inserted the note that Haskell makes "full use of literate programming". I have not been following the latest developments with Haskell and all the talk about an updated standard, but IIRC the "literate" mode most Haskell implementations support is essentially "excessive comments" together with the requirement of marking code by a prepended ">". But I still have to put "module Foo where" before any definitions in module Foo, and I cannot simply accumulate the list of exported identifiers by appending something to the list-of-exported-identifiers chunk whenever I define something. I would like someone more knowledgable about the state of literate programming in Haskell to comment on this and/or correct the mention in the article. -- Ar 16:57, 2005 Feb 7 (UTC)

I don't know about literate programming in Haskell, but I know that if it doesn't tangle/detangle, it ain't LP. Similar mistakes have been made with respect to Perl's POD. Deco 19:39, 7 Feb 2005 (UTC)

OK, I have now re-worded the Haskell mention. Ar 18:12, 2005 Jun 5 (UTC)

Link to comp.programming.literate

There is a link to the news group comp.programming.literate on the bottom of the article. However, in my opinion that group is as dead as a doornail! I am under the impression that that group is moderated by someone who does not like discussions about literate programming at all. Anyway, messages, if they come through at all, disappear very soon and when you try to reply to one, your answer is systematically sent to the moderator because you are marked as an infrequent contributor .... Not a very useful link therefore, we'd better stick to this wiki site. Bas Michielsen 22:30, 31 October 2005 (UTC)[reply]

I never had any problem with that newsgroup. There are not many people frequenting it, and sometimes noone has an answer, especially when it comes to the less often used tools. BTW, I see three posts from someone with a name similar to yours dated Sept. 27/28, so if that was indeed you the problem was not that the posts did not get through at all. (I cannot comment on cweb, though, but I don't think that LP and macros provide the same.). - Ar 18:20, 5 November 2005 (UTC)[reply]

Sorry for being late. These were questions asked by someone with the a first name identical to the first part of my last name. My suggestions never appeared on the list. The point is that with the noweb system, you can insert filters in tangle or weave pipe lines. Such filters can, in principle, be used to parse (extended-)chunk names for parameter definitions and modify the chunk contents accordingly. Bas Michielsen 01:35, 1 January 2006 (UTC)[reply]

My site

A while ago I created a web page on my web site demonstrating literate programming using Noweb, C++, and LaTeX, demonstrating a particular algorithm from CLRS. You can see it here:

Select: A literate programming example

I think it helps demonstrate the concepts in greater detail than this article, and it might be good to link. In the interest of avoiding self-promotion, I post it here instead for others to consider adding. Deco 02:25, 1 November 2005 (UTC)[reply]

Excellent. I'll check it out. -- Derek Ross | Talk 02:31, 1 November 2005 (UTC)[reply]

That's quite good, Derrick. I'll have no qualms about incorporating the link to it. -- Derek Ross | Talk 03:05, 1 November 2005 (UTC)[reply]

LiteratePrograms wiki plug

Hey all, me again. I apologise for the shameless plug, but I thought some of the people interested in this page might also be interested in my wiki, LiteratePrograms, located at http://en.literateprograms.org/. Every article on the site is a literate program. It runs on the same software as Wikipedia, but extended so that every article has a "download code" tab at the top which runs the article through noweb and spits out an archive of code that you can download, compile, and run. It was featured on the blog Lambda the Ultimate and has over 100 articles. Please stop by and consider contributing some programs yourself. I'm interested in any feedback you have. Deco 19:33, 21 April 2006 (UTC)[reply]

Changes to the program

Note that the article is also a program. While it is unlikely that edits to most of the article will have any effect on the program behaviour, one should be careful when editing any line starting with a dash. Such lines are executable code and any edits should leave the program in a runnable condition. -- Derek Ross | Talk 05:47, 21 May 2006 (UTC)[reply]

Images

I presume that images, as well as TeX, could exist in the documentation part? , for instance. � Omegatron 15:04, 21 May 2006 (UTC)[reply]

Yes. That shouldn't be a problem. -- Derek Ross | Talk 15:39, 21 May 2006 (UTC)[reply]

Relevance of the interpreter

What is the relevance of the interpreter? Is it there just to 'prove' that the example literate source code will really work? Or is it meant to give some insight into how literate compilers/interpreters might be written?

I'm not sure the first case is necessary - the 'area of a circle' code looks fairly self-explanatory; it demonstrates that you can understand what the program is doing without understanding the coding language.

In the second case, I think further explanatory text would help. Perhaps a 'literately programmed' version of the interpreter? Or at least something well-documented, describing both how and why it works. Is BASIC the best language for this? It's not as widely available as it used to be. Chris Thornett 12:22, 9 October 2006 (UTC)[reply]

I originally devised the interpreter program as an example for the interpreter (computing) article to show how interpreters worked. As a secondary goal I made it a (semi-)literate programming language interpreter so that it could also be used to illustrate this article. However someone else decided that it would be better moved to this article -- this was not my decision -- leaving a link to it from the interpreter article. It is meant to give insight into how an absolute "barebones" interpreter might be written. The fact that it interprets a literate language was secondary to its original purpose. As to a more widespread language, which one did you have in mind ? -- Derek Ross | Talk 02:38, 10 October 2006 (UTC)[reply]

Any language is more "widespread" than QuickBASIC: C, C++, Java, Perl, Python, or even (ick) Visual Basic or C#. —The preceding unsigned comment was added by 216.23.105.2 (talk • contribs).

As an addendum, I would note that changes to this article has eroded its simplicity over time. The article used to state that the whole article was a literate program -- which is still true -- but now the formatting of the article and the instructions for running the program imply that only the boxed text within the subsection Program of the section Example of a simple literate program and interpreter comprises a literate program. -- Derek Ross | Talk

Alternative languages for literate programming

BASIC

It's just occurred to me that standard line number-based BASIC forms a full literate programming system if partnered with a compiler which ignores lines that don't start with a number. The line numbers serve both to mark lines as being code rather than text and to allow for the rearrangement of code into the correct order for execution. Of course no such compiler currently exists but the language is certainly suitable. -- Derek Ross | Talk 17:07, 9 November 2006 (UTC)[reply]

COBOL

COBOL has been around since the 60's and is completely compliant with the objectives of literate programming. -- said someone who didn't sign

Any language is compliant with the objectives when the right LP editor is used, so I don't quite get your point. What I said above about BASIC could be extended to FORTRAN, provided one numbered every line. However COBOL doesn't really have provision for line numbering, so the method I talked about above couldn't really be used for COBOL. In addition COBOL's English-like syntax works against it from an LP point of view since a sequence like "ADD COST TO TOTAL GIVING TOTAL" might be a COBOL statement but it might just as easily be an English sentence. For that reason it is even more important to use some markup to differentiate the COBOL code from the English text than it would be for something like C++ which has statements that look nothing like English sentences. -- Derek Ross | Talk 04:56, 28 April 2008 (UTC)[reply]

Doxygen and Views

It is stated that doxygen cannot reorder the documentation. I think using @page and @copydoc you can do a lot. Using examples, you can use various example flows which IMHO is very helpful for readers.

Is Literate Programming ignoring "views" completely? In this case I would cosider this a serious weakness that should be noted to be objective.

With doxygen examples, "views" can be "emulated". One example program could tell "the story" from a code users point of view. It just tells how to use it. Another example could tell the story from a spezialisation library point of view. What hooks how to customize and so on. Other examples could illustrate specific usages (specific views).

It seems that in Literate Programming you can have only *one* of that views, which IMHO is simply wrong :-)

Steffen <enwikipedia-spamtrap@sws.dett.de> Jan, 2nd 2007

In my opinion that's because you understand the similarity between Literate Programming systems and Document Generator systems but you misunderstand the differences.

The relation between the two systems is analogous to the relation between a compiler and a decompiler. Both systems are similar in that a compiler and a decompiler are both translation systems but both systems are different in that one system's source is the other system's target.

In other words, Doxygen and other documentation generators are intended to be used with augmented source code to generate documentation whereas noweb and other literate programming programs are intended to be used with augmented documentation to produce source code. So the equivalent of DG "views" in an LP System would be the ability to output more than one kind of source code from the documentation.

Thus your criticism is analogous to criticising a decompiler (LP system) because it lacks the features of a compiler (DG system): The LP system may well lack the features of the DG system but that lack is irrelevant because literate programming is intended to be used to write textual articles about programs from which the programs can be extracted for validation or whatever. No more and no less. -- Derek Ross | Talk 21:33, 2 March 2007 (UTC)[reply]

As a followup to my last comment, I 'd like to point out that it would be perfectly straightforward to write a literate program article which contained source code consisting of Java with Doxygen comments. Once extracted from the literate program article, that source code could then be run through Doxygen to generate any documentation that Doxygen is capable of. So any feature (such as "views") which is available to Doxygen (or Javadoc or whatever) is also indirectly available to any literate programming system and thus does not need to be implemented as part of the LP system. -- Derek Ross | Talk 05:55, 4 March 2007 (UTC)[reply]

Literate programming languages

I reverted the sections added on "literate programming languages," which appears to be a term invented by the contributor and that have nothing at all to do with literate programming. "Literate programming" doesn't mean writing code that looks like English text - it means presenting code in an easy-to-understand way by controlling presentation order and supplying natural-language explanation of concepts that are not self-evident from the code. No programming language by itself does these. Dcoetzee 23:59, 28 April 2008 (UTC)[reply]

I undid the reversions.

Programming languages form a continuum from binary at one end to natural languages at the other. Natual languages are currently used to program humans, who can be viewed as sophisticated computers.

At the binary end of the spectrum, the burden of "literacy" in "literate programming" is all on the documentation. However, as computer languages became more natural they took up more of the burden of "literacy". That was one of Grace Hopper's goals in creating Cobol. For another example, Pascal is more of a "literate programming language" than binary, so it requires less documentation. If a compiler could parse a natural language, then "literate programming" would consist of all source code and no documentation.

Modern computer languages have been evolving to require less-and-less documentation. Python is one example. Flaming Thunder is the newest example. Here is a complete Flaming Thunder program which, within a restricted domain, requires no documenation for "literate programming" because Flaming Thunder is close enough to English that it becomes self-documenting:

   Write "Please enter a color: ".  Read color.  Write "My favorite color is ", color.

DavidBParker (talk) 01:08, 29 April 2008 (UTC)[reply]

But that misses the point of literate programming. Which is not to provide "self-documenting programs" but rather to produce "self-programming documentation". If I want to tell the world why I used the variable "color" instead of "colour"; to emphasise the importance of the word "favorite" in the above text; or to explain to future programmers that they should fix up my prompts because the only reason that I used such terse ones was that I have always been bad at social interaction, the above code is not self-documenting in the least. And it shouldn't be, because such a level of self-documentation might well impact on the performance of the program in more realistic examples. If there is a place for that level of detail it's in the documentation -- not in the program.

On the other hand it's quite straightforward to go to that level of detail in a literate programming equivalent without compromising performance or anything else. For instance the above Flaming Thunder program written as part of a literate programming article might look like this:

In many ways I had a standard Puerto Rican education and upbringing. You would probably guess that when you see that I use American spelling for my variable names as you can see in the following:

read color.

However in one way it was rather unusual. My family were rather reserved. They believed that children should be seen and not heard. For that reason I find it difficult to create good prompts. For that reason too, I would ask anyone who works on this code in the future to improve them.

Write "Please enter a color: ". <include section="InputHue"> Write "My favorite color is ", color.

I make no apologies for my decisions. However I recognize that they may not suit everyone and as a matter of principle, I urge those who work on this program in future years to change them as they see fit. However I have one request to make and that is that any changes made do not remove the word "favorite" from the second prompt. It really seems the most appropriate word to use. Thanks!

Notice that the "read color" statement was taken out of order. This is one of the advantages of literate programming which you can't get with self-documenting programs. When one is explaining the important part of the program, one can put it right up front and start talking about it. In a self-documenting program that might be a bit more awkward. Over and above that, the big advantage is that the literate programming gives the original program no matter how English-like, or machine-code like it may be, and the support files for it, and the documentation to go with it in whatever level of detail is required. In addition it can contain the configuration files and other files which are an essential part of the program but not a natural part of the code. So if we wanted to move the English prompts from the above code into a resource file, it would be easy to add the resource file to the literate program (as well as the French and German ones). It might be a bit trickier to make them part of the "self-documenting program" -- Derek Ross | Talk 05:35, 29 April 2008 (UTC)[reply]

As noted in Derek's edit summary, the important thing here isn't so much whether "self-documenting code" or code designed to resemble natural language is literate programming, but the fact that it's not literate programming in the conventional sense in which that term is used today, and unsourced speculation about future trends in LP is decidedly original research. This article shouldn't introduce concepts that are not described in the published literate programming research, or at least in widely used LP tools. Dcoetzee 07:51, 29 April 2008 (UTC)[reply]

Citations needed

I have added citation requests to the statements that have been added to this article. Wikipedia only republishes material which has been previously published, so if anyone wants this material to remain they need to come up with a publication which has made the claims independent of Wikipedia. Without those citations these claims about COBOL, or Flaming Thunder for that matter, will be removed as cases of Wikipedia:Original research. Note that citations need to reference documents of adequate quality. You can find out what's acceptable by reading Wikipedia:Citing sources -- Derek Ross | Talk 06:08, 29 April 2008 (UTC)[reply]

Related Ideas

Some work has been done by statisticians in building what they call "compendia", consisting of an article based on some data, the data itself, and programming code to munge the data. Some folks that are active in that area are Robert Gentleman, Duncan Lang, Roger Peng, Francesca Dominici, and Scott Zeger. Similar use of compendia has been made in the classification of "microarray gene expression profiles", and in math proofs (and I believe, but am not sure, that the well-known tool Mathematica directly supports this idea). Finally, I'm working on literate programming as a way of documenting grammars of natural languages. I'm not sure, though, how central any of this is to the respective disciplines (statistics, biology, math; I know my own work is not by any means a standard yet, although I'm hoping to fix that :-)).

If someone does know more about this, and whether it's worth citing, may want to add some references. (The Wikipedia article on 'compendium' does not say anything about this.) Mcswell (talk) 21:52, 19 July 2008 (UTC)[reply]

Edits which contradict Donald Knuth's original description of literate programming

Recent edits by a user with IP address 62.140.253.8/9 dispute direct statements of Donald Knuth in his papers introducing literate programming. However, these assertions are not sourced. While it may be claimed that there are additional benefits from literate programming, as propounded by the anonymous editor, I am not aware of any recantation by Mr. Knuth of his original presentation. If there has been one, I would appreciate a reference to it. In addition, any claims of additional benefits of literate programming should be referenced to verifiable sources and should not rely on original research, in keeping with Wikipedia's policies. HowardBGolden (talk) 01:21, 23 December 2008 (UTC)[reply]

ANSWER TO ABOVE from the author of the new version: WRONG!

It is I who (after changing the article and when, absolutely predictably for Wikipedia my changes were rolled back almost automatically, the first knee-jerk reaction) pointed that I introduced changes BECAUSE the previous version was INCORRECT TECHNICALLY and CONTRADICTED the plain pronouncements of the creator, D. Knuth

You can see it recorded in the history page

Now, to kind of jump ahead with accusations YOU manipulatevely throw back my point at me and try to cover the change war with "look, he dares disputes Knuth"

Moreover, you used another irritatingly low-brow method of inserting "citation needed" after (incredibly) every sentence of my text.

Probably you do not realize the position you put yourself into. Imagine someone posts a photo of a cat and proceedes to describe it as an "animal with four paws and a tail"; your notes are equivalent to demanding some citations to confirm of each of the used words, cat, paw, tail and four. This is insanity, of course, unless understood as a tool to cover with fake "objectivity" the fact of venting one's irritation.

Each of my major assertions is self-evident: that "literate programming" is a PREPROCESSING MACRO SYSTEM (which Knuth named as such, something like "macros without arguments"); that they stand for arbitrary abstractions, that they can build on each other, and that the main thing is to CHANGE ORDER to that of thinking (Knuth actually wrote of "stream of consciousness", i.e. continuity of human thought as contrasted with continuity demanded by the machine.

To see the truth of it it is sufficient to stop forming one's opinions on the talk "about", L.P. and READ THE direct EXAMPLE OF L.P. to which I referred THREE times.

Of course, some readers might be unused to following connections in a text and need "soft" explanations, but by the nature of it an article on Literate Programming is not likely to attract "students of interior design" and similar types.

SECONDLY, upon some consideration I decided to withhold my irritation and treat your reaction more as that of astonishment at the encounter with the real meaning of L.P. paradigm - in view of that very misconception that rooted itself so strongly in the public mind.

I will re-write the article, almost in full, composing it from Knuth's own pronouncements as much as possible, and instead of simply referring the reader to an example, I will include a couple of snippets from it to directly illustrate the two major points.

It's much, much better to keep all explanations in one place, especially if they are essential. —Preceding unsigned comment added by 62.140.253.8 (talk) 20:54, 24 December 2008 (UTC)[reply]