Help talk:Template limits

New template limits, Special:ExpandTemplates

The new template expansion limits, announced on wikitech-l and wikipedia-l, are now in effect on a trial basis. Keep an eye out for any broken articles. Note that there is some information to help track down problems in comments in the HTML source of the parser output. To help editors to substitute templates for literal text in problem articles, I've introduced a new special page: Special:ExpandTemplates. It works like adding subst: to all the templates, except you don't have to repeatedly save the page. -- Tim Starling 07:41, 14 August 2006 (UTC)

Example [1]: Shows a test with inclusions of template:cite web, where the limit is reached at 267 example calls of cite web. The critical thing is the Pre-expand include size, which is now limited to 1MB (Look at the html source of the page and find the string "Pre-expand include size", which is included in a html comment). The html contains {{cite web}}<!-- WARNING: template omitted, pre-expand include size too large --> for calls that exceed the Pre-expand include size limit. --Ligulem 09:13, 14 August 2006 (UTC)

From the above link: "Example Shows a test with inclusions ..."
[http://en.wikipedia.org/w/index.php?title=User:Ligulem/work/sandbox&oldid=69549203] 
<!-- 
Pre-expand include size: 918900 bytes
Post-expand include size: 218100 bytes
Template argument size: 238200 bytes
Maximum: 2048000 bytes
-->

I'm not sure where to discuss this, but I have noticed several pages that use "Template:Familytree" include that template a number of times, and when it gets to about 26 transclusions they aren't expanded any more. Here's an example: User:Timc/Sandbox. --  timc  talk   02:54, 15 August 2006 (UTC)

Template:Familytree is large and ugly. You can either reduce its size, to allow you to fit more of them in the 1MB limit, or you can subst it into pages instead of including. -- Tim Starling 03:33, 15 August 2006 (UTC)

Tim, how does the limit work with things bounded by <noinclude> tags (or everything not bounded by <onlyinclude> tags) on the template page? Do they add towards the limit or does only the info which is actually transcluded for evaluation counted? I'd assume they aren't included in the computation, but worth checking. Also, {{listadmins}} fails now, but lists the Pre-expand include size as 'only' 837703. I assume it stops counting at and doesn't include the item that puts it over the limit? --CBD 10:55, 15 August 2006 (UTC)

I'm not sure, but the relative size of the <noinclude> section is small compared to the rest of the template. --  timc  talk   13:04, 15 August 2006 (UTC)

<noinclude> sections are included in the pre-expand size. If the documentation is large or frequently changed, I'd recommend that you move it to a subpage. Then you can transclude it into both the <noinclude> section and the talk page. It does indeed stop counting when an item puts it over the limit, this behaviour is intended to allow the bulk of a page to render correctly when there is a single large template present plus a number of small templates further down. The large template will be left out, but the small templates will still be included. -- Tim Starling 21:18, 15 August 2006 (UTC)

Indeed. If I remove the <noinclude> part from template:cite web entierly and repeat the test I described above here in this section, I can do 380 inclusions of the cite web template code instead of 267 for that testcase. What a difference! I would strongly recommend that an admin edits template:cite web accordingly. (Cite web is currently included on 18,500 pages) --Ligulem 22:49, 15 August 2006 (UTC)

I am glad I found this section, I was wondering what is happening to Wikipedia:Copyright problems. In this page, we transclude sub-pages for daily reports of copyright problems; everything was going well, but we currently have quite a large backlog, meaning that we seem to hit the limit. Any idea about what we should do ? Remove the backlog would be a good thing to do (but I can't help much with that), but it will come back at some point. Schutz 22:54, 15 August 2006 (UTC)

I would say you need to completely redesign that page. Transcluding that much of stuff into a page is no longer working. --Ligulem 22:58, 15 August 2006 (UTC)

Hum... Ok, we could simply link to the pages for the individual days instead of transcluding them (the way it is done on WP:AFD); that would solve the problem in the short term. However, it could become a problem for pages such as WP:AFD too; looking at one random but recent example, Wikipedia:Articles for deletion/Log/2006 August 11, this page is getting close to the limit. In a "good" (or rather bad) day, there may be many articles listed for deletion, with a few of them attracting a fair number of comments, and it may be enough to get past the limit. Is the 1 Mb limit negotiable if we see that some of these articles get bitten by the limit ? Schutz 23:23, 15 August 2006 (UTC)

The limit is negotiable, maybe we could raise it to 1.5 or 2MB if there was a solid case for that. But note that splitting these pages up does have a significant beneficial impact on server load. It might be better to analyse them and look at why they get so big -- maybe it's a big hassle to manage per-day pages like on AfD, and even more so to split them up further. Maybe new features in MediaWiki or bot-driven automation could help with that. I'm willing to hear your ideas. Increasing the limit might be good in the short term, but it's not going to solve the usability and performance problems that huge pages cause. -- Tim Starling 00:53, 16 August 2006 (UTC)

Wikipedia:Articles_for_deletion/Log/2006_July_28 hits the limit with 13 debates un-transcluded. Pre-expand include size: 1047585 bytes Post-expand include size: 956640 bytes Template argument size: 4759 bytes Maximum: 1048576 bytes Wikipedia:Articles_for_deletion/Log/2006_July_26 is close, don't know about anything beyond 20060810224009 though.

The number of pages on AFD is only going to keep increasing, and I can see how the pages may not be managable for people as is... so I see how just increasing the limit doesn't really solve anything. However, can you have the template parameters listed and/or the transclusion linked? Would make AFD pages being cut off at least bearable still (but WP:AFD/Old's statistics and other bot run things are probably broken) until things are sorted out. Kotepho 01:44, 16 August 2006 (UTC)

Having omitteded transclusions linked would be nice to have, but not critical, as those pages need to be fixed anyway by editors. I don't know how ugly it would be to implement in MediaWiki. If it is ugly to implemenet, I would say leave it as it is. --Ligulem 08:03, 16 August 2006 (UTC)

For WP:CP, as mentioned above, I am going to make subpages instead of transcluding so many pages; this should solve the problem (there aren't that many copyright violations). For WP:AFD, I don't know, but I am happy to help with any bot work (especially given that I found the problem on WP:CP while investigating my bot's work). Schutz 06:46, 16 August 2006 (UTC)

I'm not terribly familiar with what y'all are talking about in this thread, but is this what's going on in List of Hebrew names? —ptk✰fgs 03:51, 16 August 2006 (UTC)

Yes. If you look at the source, you will see errors like "<!-- WARNING: template omitted, pre-expand include size too large -->". --  timc  talk   04:16, 16 August 2006 (UTC)

I've tried to subst template:unicode but that produced a section that looked like this (excerpt):

{{{1}}} {{{1}}}
    * {{{1}}} {{{1}}}, {{{1}}}, Tola. "Worm; grub." Masculine.
    * {{{1}}} {{{1}}}, {{{1}}}, Thomas. "Twin." Masculine.

Substing just some templates on a off-limits page doesn't seem to work. It looks like we can't use MediaWiki server for substing in this case (yes we could subst everything by Special:ExpandTemplates, but I wanted to try substing gradually). --Ligulem 08:44, 16 August 2006 (UTC)

I fixed List of Hebrew names by substing template:unicode in several steps (per sections) in my sandbox. --Ligulem 09:05, 16 August 2006 (UTC)

Tim, above you suggested transcluding documentation into a 'noinclude' section of the template page rather than having it reside there directly. That would seem to imply that the 'pre-expand' size of a template is only the size of the contents of that page NOT including the size of any templates called by it... otherwise there would be no benefit to the procedure you suggest. This almost seems to then become an argument for 'meta-templating'. For instance, wouldn't that mean that if an article contained {{example|param1=Fred|param2=Fish}} that 'template:example' could be set up to be just {{handoff|{{{param1}}}|{{{param2}}}}} with all of the complicated logic/text actually in the 'handoff' template? Thus minimizing the size of the template called while still having exactly the same logic/display - just once removed? Obviously, that's... silly... but I'm trying to understand the rationale behind the methodology here. We have been consolidating alot of things with sub-templates into single templates to avoid 'nested transclusions', but this change now seems likely to lead to the reverse.

Would it somehow be possible TO include 'sub-template' size in the 'pre-expand' size and NOT include 'noinclude' type materials? Since the goal (as I understand it) is to limit the amount of text being 'transcluded in for evaluation' I would think we would want to take these issues into account. For example, if a 900kb page is transcluded, but everything except the word 'frog' is bounded by 'noinclude' tags you are really only transcluding the word 'frog' and the evaluation is quite fast (I just checked)... so why count the full 900kb towards the limit? In contrast, if you call a template which is only six characters long, but those six characters are calling another page which is 900kb long it takes several seconds to evaluate but still only counts as six characters. I can imagine that there are technical difficulties in ignoring 'noinclude' info and including 'sub-template' info, but not doing so seems to give some results wildly inconsistent with the intent - and open an easy loophole for getting around it while page loads are still just as slow. --CBD 00:47, 17 August 2006 (UTC)

Sub-templates are included in the size, just not sub-templates included from within <noinclude> sections. To put it another way, the <noinclude> sections are counted and removed, but they're not expanded. There's no loophole. Documentation subpages have been an agenda of mine for months, because at the moment, editing the documentation for a template adds an item to the job queue for every page that uses the template, and clears the cache of all those pages too. Documentation subpages are not registered in the links for pages using the template, so editing a documentation subpage only clears the cache of the template itself, not the pages that use it. There is a technical solution for this: some people have suggested stripping <noinclude> sections from the template before and after a change, comparing the two, and then only clearing caches and adding items to the job queue if the resulting stripped version has changed. It'll probably be implemented at some stage, but it's not here yet.

As for removing <noinclude> sections from the limit -- I did consider it when I was writing the feature. Although <noinclude> sections are among the cheapest things the parser has to process, they still do have a finite cost. One could easily imagine a DoS attack based on including thousands of templates consisting of only a 1MB <noinclude> section -- or worse, tens of thousands of small <noinclude> sections per template inclusion. Whether or not it has <noinclude> around it, the text still has to be loaded from the database. There are costs such as disk I/O, DB cache size, network bandwidth, and of course the CPU time spent removing the tags. It seems to be a good candidate for inclusion in the limit. -- Tim Starling 05:57, 17 August 2006 (UTC)

I've been thinking about doc subpages for templates too, because people do like documentation on template pages and editing of documentation should be separated from editing template code (at least for heavy use templates). I image we could even have a doc tab for templates (we currently have "template" and "talk"), but I have no idea how ugly this would be to implement. Yet another namespace? In any case, it is very interesting that you are thinking of documentation subpages. --Ligulem 09:02, 17 August 2006 (UTC)

Ok, that (only sub-templates in 'noinclude' sections are 'not counted') makes alot more sense. My primary objection to counting 'noinclude' sections is that I had just been discussing a change to featured lists to mark various sections of a list page 'onlyinclude' so that when the list page were transcluded it would produce a nice little blurb similar to an 'article of the day' page (see User:CBDunkerson/Sandbox2 for an example)... which could then be made into a 'List of the day' and/or 'random list' on the featured content page. Obviously, that is now untenable and separate pages would have to be set up for the blurb on each list and manually updated when the list info changes. While the 'resource cost' of unincluded sections is not zero it is clearly significantly less than the cost of included sections. If there are not significant technical difficulties in doing so it might be nice to somehow reflect those different impacts in the 'expansion limits' logic. --CBD 10:42, 17 August 2006 (UTC)

If it aint broke, don't fix it

In relation to this template expansion limits issue I'd like to suggest that people hold off on 'fixing' things for the time being. If a page isn't displaying right then obviously we need to do something to get it back to working, but I'm seeing people mass moving documentation, substing templates, discussing redesigns, et cetera. We may end up adopting standards and wanting to do these things... but there is no rush. If it isn't causing a page to display improperly right now then changing things before this new limit is fully evaluated/understood and worked into our standards may end up needing to be changed back. Until yesterday it would have been inconceivable to have a 'documentation' sub-page transcluded in for every template (or many templates). Before we do that to hundreds of pages I think we should evaluate it a bit more. It might be easier to just make documentation on template talk pages standard (though I've always preferred it on the template page). Or there may be changes (per my questions above) which make this documentation problem moot. And there are other issues to consider. For instance, note that categories and interwiki links shouldn't be relocated to such a sub-page... which means a popular template that is interwiki'd all over actually becomes less usable because of the increased size from the interwikis - so maybe we want to consider a different way of doing template interwikis. Et cetera. --CBD 01:11, 17 August 2006 (UTC)

In case you are referring to my edits (example): The benefits of the doc subpage for in template included noinclude parts is obvious. Templates which are by their nature intended as meta-templates (templates to be included in other templates), like those from Category:Date computing template or stuff like template:cite news which is intended to be included in pages multiple times (10, 20 times) clearly benefit from the extracted doc pattern. The MediaWiki parser has to parse the whole content of the noinclude, but it can skip templates included in noinclude sections. And per the change on template cite news: if we leave the doc inside the template page itself, it "eats up" from the pre-transclude-size of each page this template is included. Wikipedians will stop using templates like template:cite news if they encounter that the limit is crossed at the very moment they look at a broken preview of their edit, and they will skip that edit. Why should we risk that, if we know better? So what do you want to wait on? What do you need to "evaluate" further? What isn't understood? --Ligulem 07:38, 17 August 2006 (UTC)

What does discussion / evaluation help? Well, until I asked about unincluded sections people were talking mostly about recoding templates to save space... since they have been talking mostly about removing documentation. Until I asked about sub-templates it wasn't clear (to me at least) that while unincluded sections are counted sub-templates in unincluded sections are counted by the length of the template call rather than the template page... though the reverse is true for included sections (i.e. page size not call size).

What is there still to discuss / evaluate? Well, if we are going to do this 'documentation sub-page' methodology we should think about how best to work it. For instance, in your 'MONTHNAME' example above you left the categories and interwikis behind on the template page when you created the documentation sub-page. However, in my experience those are the two unincluded items which get updated most frequently. After thinking about it I wonder if it would make sense to also move these to the 'documentation sub-page' and place them inside 'includeonly' tags there so that they show up on the template page, but not the sub-page? Any drawbacks to that? Would it be a good idea to establish a standard that the 'documentation sub-pages' of protected templates should be unprotected so that anyone can add interwikis and the like? 'Look before you leap' is seldom a bad idea. You reference how removal of unincluded sections (arising BTW out of the discussion here) has increased the 'cite-news limit' from 267 to three-hundred something. Ok, but.... were there really any pages with more than 267 news citations on them? Or running over the limit because of this template in conjunction with others? It seems implausible to me... so what harm in fully evaluating and discussing the best way to handle this new limitation?

Sorry, I'm a 'ducks in a row' kind of guy. I like to get all the answers and general agreement on standards before implementation. Ohhh... another idea, could we standardize on using something short like <noinclude>{{/doc}}</noinclude> for all 'documentation sub-pages' so they can always be found at the same name? Also, if we made a ==Documentation== or somesuch standard at the top of all sub-pages then people who have 'section editing' turned on would see an edit link for the documentation on the main template page, but actually be editing the sub-page when they clicked it... rather than having to know how to find the sub-page at 'Name/doc'. --CBD 11:09, 17 August 2006 (UTC)

Ok. I'm more the quick shot one :) (If I see there is no danger in doing so). BTW, I was fixing [2] by applying the doc subpage pattern. But I will hold off from further actions for now (but I do think I already catched some of the worst cases anyway). --Ligulem 11:28, 17 August 2006 (UTC)

Found a casualty on meta

http://meta.wikimedia.org/wiki/Template:List_of_language_names_ordered_by_code --NE2 16:33, 17 August 2006 (UTC)

Fixed by replacing the template calls with the expanded wiki-source by using m:Special:ExpandTemplates (see project page). --Ligulem 18:36, 17 August 2006 (UTC)

Not very useful

Forgive me, but this isn't a useful feature, particularly without giving any kind of warning on Wiki (the mailing list is all well and good but basically invisible). It breaks WP:CP for a start, so the WMF can just live with noone being able to list, review or delete illegal text. Or, the devs can write us a bot that daily subst's the page. Or just get rid of this 'feature'. -Splash - tk 17:24, 17 August 2006 (UTC)

"It breaks WP:CP for a start", well nothing new (see VPT), that's why I've added it to the new Category:Pages exceeding template inclusion limits. And I must say we shouldn't optimise MediaWiki for this kind of page. It is very atypical. I'm sure we will find another solution for this page. And the "feature" is not a feature per se, it helps running the servers smoothly and disables certain template DOS attacks. These limits make sense to me and I trust Tim that he installed it for good reason. And I also see no problem the way he introduced it. I have read the announcement on the mailing list and I was aware what he intended to do. And you can't get your feet wet by endless theorising upfront. These new limits do work astonishingly well. --Ligulem 18:21, 17 August 2006 (UTC)

Of course it's new. CP wasn't broken until this was introduced. Adding it to a category does what, exactly? Does it allow AfD to function? CP? Does it break articles, processes, policies? And I suppose we're supposed to accept that, say, everyone reads VPT? I didn't ask for theorising, I asked for some advance warning that large numbers of things were going to be wilfully broken. In what sense do they "work astonishingly well"? They are invisible unless they break things, at which point things aren't working at all. I read the mailing list posts now and see that they are protection against a problem that never occurred, which is all very well, but it's a solution that is really a long way suboptimal. -Splash - tk 19:06, 17 August 2006 (UTC)

Yes it's new. What I wanted to say is, it had already been reported on VPT. As such, it's no news. Which "large things are willfully broken"? BTW the problem *did* occur here, caused by such things (which demonstrated the attack vector). Or do you think server response times in the order of 10 or 20 seconds to update the cached html of a page are acceptable? Well, ok, I'm not a WikiMedia dev anyway and do have nothing to say about how this wiki is configured. Tim does. He's also co-responsible for keeping the site running. So probably I should better shut up on this anyway. But Tim has my support on this. --Ligulem 23:13, 17 August 2006 (UTC)

Limit increased, links added

I increased the limit from 1024KB to 2000KB and changed the code to add links to missing templates. WP:CP now works, although it takes 8.5 seconds to render and generates 1.5 MB of HTML, so I wouldn't like to call it "fixed". -- Tim Starling 23:31, 17 August 2006 (UTC)

Thanks. Sorry for the edit conflict (did I overlook an indicator?), I just happended to have rechecked WP:CP at the same time. Yes, the page should and can be reorganised, iff Splash (and others) agree to reorganise this page. --Ligulem 23:43, 17 August 2006 (UTC)

Don't worry, the complaint in my edit summary was in jest, it's not late enough in the day yet for me to get angry at edit conflicts :) -- Tim Starling 23:54, 17 August 2006 (UTC)

Yes, that page will inevitably exceed the limit again before too much longer if the methodology isn't changed. Though they have plenty of warning now. The links for suppressed templates is a nice addition BTW. --CBD 23:50, 17 August 2006 (UTC)

I imagine "they" would be very open to suggestions about how to change "their" methodology if you'd like to give "them" a call. Although honestly, the only changes "they" can probably make are to not display the entries on "their" page. A methodology that doens't actually display the listing isn't so hot, really, but perhaps it's the only way if whatever the actual problem in MW is can't be fixed further away from the editorial frontline.  -Splash - tk 00:53, 18 August 2006 (UTC)

Please replace "they" with "we" :). Seriously: Splash, thank you for expressing your concerns with WP:CP. Now let's try to find a solution: Do you think it would be completely inacceptable to split up the page in subpages, and simply link them from WP:CP instead of transcluding each and every subpage into WP:CP? Please forgive me, but — while certainly very important for organising the project — WP:CP is not part of the encyclopedia material as such, so couldn't we try to be a bit less demanding there and save some resources in favor for computing power for the articles? --Ligulem 07:36, 18 August 2006 (UTC)

It is less a concern with CP and more a concern with the 'solution' to the problem taken by the devs. (Who can of course solve problems however they choose, but they can still take criticism, too.) It appears to me that the 'solution' has been to 'fix' MediaWiki by breaking Wikipedia, and so the 'solution' is a bad one. I think you're over egging the pudding rather with the implication that CP is causing computational resource problems with article reading/writing/maintaining. Both were getting along dandy until this new (arbitrary) limit was invented. As to your suggestion, well, that's what I had in mind when I said above that a methodology that doesn't display the listing isn't so hot, but may be the only way. CP is a ghastly backlog because it's damn hard admin graft -- making it the harder is only going to work around the (new) technical flaw rather than actually make anything better. But I can't see any other way until MediaWiki is fixed. -Splash - tk 11:19, 18 August 2006 (UTC)

I should be clearer on why non-transclusion is a poor approach. Apart from the admin-hate it will generate, it will make it rather much the harder for an editor to find their article's listing if it doesn't appear on the page the relevant links take them too. ({{copyvio}} isn't substed becuause....it's so much code, oh the irony.) Having to negotiate their way through several potentially large pages to locate their item is going obviously to cause them problems. Now not many copyvio listings are challenged, but really MediaWiki shouldn't have aspects that actually make life harder where before it was relatively easy. -Splash - tk 11:26, 18 August 2006 (UTC)

Well, it wasn't easy for the servers and Tim probably had to decide to do something about it. He sure can't wait until the servers are down before he does something. Furthermore, we probably cannot expect that pages like CP are explicitly excluded from the template limit logic. I know we editors are damn greedy when it comes to features (I was such a bad guy myself on that qif debacle) and the devs do have to give us some firm stops when needed (preferably done in the software and not with policies). And it is up to Tim to decide what stops are needed. Wikipedia is growing tremendously and it's in the interest of all of us that it runs well. Some collateral "damage" (from editors standpoint) is sometimes inevitable. As long as only some procedural pages like CP are effected, we are better off taking that bitter pill for the sake of the whole site. Ligulem 12:00, 18 August 2006 (UTC)

See Wikipedia talk:Copyright problems#New template limits. --Ligulem 12:28, 18 August 2006 (UTC)

No, it really is 'they'. The regulars at CP are the ones who will have to deal with whatever system is used there. As to suggestions... to keep generally the same functionality while reducing transcluded size I'd go with a change of archiving method. Currently alot of closed entries stick around for weeks until all the other entries from the same day are closed. If you instead had a monthly archive page (with headers for each day in the month) you could move each entry to the archive as it was closed and then delete the empty 'day' pages when they had been fully resolved. More manual archival process, but clears out closed discussions immediately and consequently reduces the size of the page. Making it easier to follow as a side-benefit. --CBD 19:46, 19 August 2006 (UTC)

There aren't any regulars at CP. Generally, 'closed' entries are removed from day listings as they are dealt with, and links that are blue are genreally un-dealt-with. This means that the day pages decrease in size as they are cleared. They don't get deleted because it then makes it hard for people to go and work out what happened to a given listing. Minor points aside, I'm not quite with you on how this reduces the size of the pages that don't transclude in the first place while leaving them visible; nor how it fixes the (should-be-non-) problem with the "current listings". But maybe I misunderstand something. -Splash - tk 21:46, 21 August 2006 (UTC)

let us remember

the limit is quite high, plain transclusion like used on the deletion pages etc really shouldn't exclude them without making the page insanely big, its when structures are used that recursively call templates or templates with large noinclude sections that problems may occour. Plugwash 13:12, 21 August 2006 (UTC)

Hmm, I'm sorry, but I don't completely understand what you want to say with your post (scratching my head ;-). Per the "quite high": WP:CP did hit the 1MB limit until I changed some transclusions to links (a reorganisation is still pending there). AIDS is currently at 363,088 bytes pre-expand size. So we can't completely ignore these new limits. Although 2MB is quite comfortable. And the new doc template pattern is neat anyway (example: Template:MONTHNAME). Templates that are called many times on the same page are also adding pretty much to the pre-expand size. Cutting down on hefty noinclude content on these templates by using the doc pattern is really beneficial. --Ligulem 17:49, 21 August 2006 (UTC)

Salaam

• Lower the limit to about 512K. Gigantic transclusions are signs of something that's already broken. For example, WP:CP is already broken, from a social perspective; the backlog is enormous. Instead of catering to the failure, better to force a rethink, since nothing else has prompted one.

• Transcluding template documentation onto template pages via noinclude is just plain silly. John Reid 18:11, 31 August 2006 (UTC)

You contradict yourself. Could you explain please? What exactly is "silly"? And why? What does the heading "Salaam" mean? --Ligulem 21:55, 31 August 2006 (UTC)

Salaam is Arabic for 'peace' and a traditional greeting. As to decreasing the limit to force things like WP:CP to slim down... it wouldn't work. The size of that page and others like it is a factor of Wikipedia's rapid growth. A smaller limit on transclusion would just force those pages to include the full text directly... no transclusion making the limit irrelevant. As to 'doc sub-pages for templates. With the details Ligulem has incorporated they actually work out quite well, especially for heavily used templates that are permanently protected. --CBD 09:00, 3 September 2006 (UTC)

<Off topic> I do know the word "Salaam", thanks ;-). What puzzled me is why it was used as a topic for a discussion. First "Salaam", and the "plain silly" ;-). What a contrast. But nevermind. --Ligulem 09:27, 3 September 2006 (UTC)

Idea for reducing size used by documentation/usage

Maybe what is needed is a "Template usage" namespace, which could be used for documenting the template and parameters along with example usage (and test cases, if needed).

I've tossed around this idea on other another wiki, but decided that with template limits being enabled it might be time to file a bug with the suggestion. (See bugzilla:7210). I'd like to know what others think of this idea. Would it be confusing to have a third tab for templates? —TheMuuj ^Talk 21:33, 2 September 2006 (UTC)

Good idea. Did you read Wikipedia:Template doc page pattern? We can already do a lot with that a the moment (See template:cite news for an example). --Ligulem 22:38, 2 September 2006 (UTC)

That's a pretty good approach. I'll be sure to use it in the future. —TheMuuj ^Talk 17:36, 3 September 2006 (UTC)

Parsing problem

I had a problem with 1.8.2 which was fixed by removing the automatically-generated six-line HTML comment. See Cite.php page on meta.wikimedia.org. I can provide more detail if necessary, as I accept that my message on that site was a bit garbled! Best wishes Jonathan3 11:44, 7 January 2007 (UTC)

Confusing documentation

I hit the template limit recently on a page using templates to list mathematicians by date. I was directed to this article for an explanation, but I found it so confusing that I even thought it was self-contradictory! Judging from other comments on this talk page, I may not be alone. Anyway, I think I understand the issue better now, so I hope previous editors won't mind if I try to rewrite some of the article to clarify it for the benefit of non-experts. I will do my best not to introduce any incorrect statements, but I trust the experts to keep me honest ;) Geometry guy 17:48, 8 May 2007 (UTC)

I rewrote this page this afternoon; I hope the new version is more clear. If any developers are watching this page, please feel free to correct any errors. I looked through Parser.php and tried some things in my sandbox to find out how it works, but it's possible there's some subtlety that I missed. CMummert · talk 23:52, 8 May 2007 (UTC)

Thanks! I've gone through it adding more structure and explanations in the light of comments that other editors have left on this page. Again, checking by experts and developers most welcome. In particular I'm not sure whether I'm correct to say that the length of the HTML is the figure used by the post-expand counter. Geometry guy 17:06, 9 May 2007 (UTC)

Template limit problem

I'm currently involved in helping to re-write the article general relativity, more specifically: in making a work-in-progress/sandbox version at Talk:General relativity/WIP for later inclusion in the article. Since we eventually want to bring the article to FA status, we're putting a lot of work into finding proper references, and since it's a complex topic, there is quite a number of references to be included for proper documentation (even if the article itself is, naturally, summary style, with lots of spin-off articles). I now appear to have hit the template limit, with a hundred-something references included, and the list of references not nearly complete (the sandbox so far contains only one of the article's sections, after all). From previous postings here, I gather that one attitude towards this is "if it has so many templates, it needs to be fixed anyway". In this case, I'm not sure how this is supposed to work. I'd certainly be loathe to throw out references and thus compromise the article's content to satisfy some technical limit that, as far as I can see, is rather arbitrary. Did I make a technical goof somewhere? Am I using more templates than I could be? If not, is there any way the template limit can be avoided, or changed? Any help on this would be greatly appreciated. --Markus Poessel 08:08, 17 August 2007 (UTC)

For the citation template the pre-expand include size is about 18500 bytes, so without any other templates you can call it a little over 100 times. You could do substitution (not only of the outer template but also of the core template and the ifs).--Patrick 09:32, 17 August 2007 (UTC)

Many thanks for your quick reply. A little over a 100 times sounds about right - though I've no idea why whoever set the limit thought that would be enough. While I've been using these templates, I know very little about how they work. I take it that by "substitution" you mean that I use Special:ExpandTemplates to transform the Citation thingies into something more fundamental? I have no idea what you mean by expanding not only the outer template, but also the core template and the ifs (unless ExpandTemplates does that automatically?); any pointers on where to read up on this would be appreciated. --Markus Poessel 14:49, 17 August 2007 (UTC)

Yes, ExpandTemplates does that automatically, the other way is using "subst:" (see m:Help:Substitution), but that seems more cumbersome here.--Patrick 00:08, 18 August 2007 (UTC)

Thanks for the helpful information! --Markus Poessel 15:59, 18 August 2007 (UTC)

I recommended to a friend to make a list of his used literature to have it ready for good citations in a lot of his articles.

I asked him to use the German citation templates to have the state of the art.

We were stroke by the limit and at first we wondered why.

The german template's comments were outsourced to save some bytes but we are still in trouble.

• de:Benutzer:JEW/Literatur

Is there a chance that some bytes more will be allowed? I find this neccessary for a appropiate form of working in Wikipedia. Thank you very much! -- Simplicius 01:47, 8 November 2007 (UTC)

New preprocessor

Thanks for the updates to this page post the new preprocessor implementation. Can someone please explain how the preprocessor node count works? Geometry guy 11:59, 26 January 2008 (UTC)

Since I'm still waiting on the toolserver db to come back up, I'll look through the source and fix something up. Patrick did a nice job of trimming out the stuff that is no longer relevant, and correcting my date error. I think that the next step is to think about whether there is a better structure for the page given that the new limits are easier to understand. — Carl (CBM · talk) 14:55, 26 January 2008 (UTC)

Nice rewrite! One query: is it really true that all template arguments are fully expanded and their length added to the template argument size? This would imply that ignored #if's and #switches could still add a lot to the template argument size, whereas they don't seem to in my (admittedly anecdotal) observations. Geometry guy 22:11, 26 January 2008 (UTC)

Actually, as I have been playing around with things, I see that I don't understand everything as well as I thought I did. I'll let you know when I figure out what's going on. — Carl (CBM · talk) 22:39, 26 January 2008 (UTC)

Why does is UTF produce an error but not a template that produces the same UTF?

Resolved

OK it is against expectation. I am writing a template to produce possible Arabic verb forms from the verb root. I have run into the parser limit (the HTML source says so). Because Arabic is bidirectional and messes-up things when written within left-justified text, I created a subtemplate for the letter ta instead of using the unicode value for readability reasons. This worked partially, but but didn't parse for the more complex verb forms. So I tried to reduce template transclusion in my code by using UTF values instead. The strange thing is that even the verb forms that worked before don't work now. The current version is the older version with the {{/ta}} template. How can that be?.(I tagged the problematic code with ERROR HAPPENS FORM HERE)-- hɑkeem_¡ʇu^ɐɹɯǝǝʞɐɥ 10:08, 21 May 2008 (UTC)

I was #ifeq'ing utf values with characters input from the keyboard. I learned that wiki doesn't process utf's but passes them to the browser. Thanks. -- hɑkeem_¡ʇu^ɐɹɯǝǝʞɐɥ 05:25, 22 May 2008 (UTC)

Proposed use of a date metatemplate into citation templates

I'm shortly hoping to introduce optional date formating into the cite_XXX family of templates, with cite_web acting as the test case - see Template talk:Cite web#Working version and final discussion . In essence the various date parameters (date, archivedate & accessdate) may be optionally formated into non-linked dates (as per changes at WP:Dates) by use of a metatemplate (see proposed new coding at {{cite web/sandbox}} and its use of {{date style}}). Whilst I've been advised not to worry about use of metatemplates, I'm not so sure that is true, as cite_XXX templates are not used just once or twice in articles but sometimes a very large number of times and across a large part of all wikipedia articles.

So will formating citation template dates, using metatemplate {{date style}}, cause a problem (if so would very ugly repeated direct date-format coding within {{cite web}} help)? Further, would using a meta-metatemplate to simplify the coding in {{date style}} help or hinder ? David Ruben ^Talk 14:28, 17 July 2008 (UTC)

Why don't you just edit the date regexes in DateFormatter.php so that it matches unlinked dates? Then you wouldn't need templates. -- Tim Starling (talk) 13:46, 19 July 2008 (UTC)

Sorry, I don't understand PHP, so that totally lost me :-( Both that I'm unfamiliar with PHP coding, but also that is a onetime edit for the whole of wikipedia if I understand correctly, but there needs be flexibility for American or British/European/International date formats which would need be selected on an article-by-article basis. (or am I missing something here and wikilinked-dates do allow a formating-style parameter to be included ?)

The three wish-list items are 1) show user-preference style if this has been set (minority of editors) 2) else select a date style appropriate for the article in question 3) in eithercase show as an unlinked item. The later is need to follow WP:Dates now stating dates should not appear wikilinked.

So whilst wikilinking dates switches appearance of input for those few editors who have selected a dmy/mdy/ISO preference, all the output (for those with & without a preference set) is still blue, underlined and linked.

Now if MediaWiki allowed a magic-word like feature of the same input flexibility (unlike #time parser function's limited date range) but without linking then I agree no problem. Not sure how one might so flag that, perhaps use {{foo}} or perhaps if need to help the system recognise a date as {{@foo}} ?

Alternative or addition options for MediaWiki assistance with this might be:

• Allow reading of a user's preference option (if no option set then editors could decide which style best suited the article in question)
• Allow article variable on this (we have some magic words that MediaWiki recognises for its own use) so we might have __DateDMY__ or __DateMDY__ or equivalent {{DATEdmy}} or {{DATEmdy}} and then mediawiki follows that preference if no user preference already been set.

For now, I'm not sure whether/how to proceed. David Ruben ^Talk 14:38, 19 July 2008 (UTC)

what are the limits that cause "Category:Pages where template include size is exceeded?"

• i understand the need for this, but was wondering how is it triggered and/or configured? Bud0011 (talk) 05:47, 21 June 2009 (UTC)
• Me, too. How and where I can change the settings in any MediaWiki-Installation (not wikipedia)? --EnterpriseWiki (talk) 11:39, 29 July 2010 (UTC)
  • bumping this.Bud0011 (talk) 05:29, 27 November 2011 (UTC)
  • dito — Preceding unsigned comment added by 114.198.35.204 (talk) 07:10, 21 March 2012 (UTC)

You might like to look at Wikipedia:Avoiding MediaWiki expansion depth limit. The question is very technical though, and you'll probably get a better response at either meta:Meta:Babel or mw:Project:Support desk. --Redrose64 (talk) 14:44, 21 March 2012 (UTC)

Yeah, well...

If there are going to be such template limits, then there should be limits on templates being unnecessarily expanded/rewritten with no regard to bytage; one consequence of a thoughtless substitution of a "new, improved" template ({{cite bcgnis}}) for a completely easy-to-use and less-bytage one ({{BCGNIS}}, although what's at that link now isn't waht it originally was thanks to its predecessor being "deprecated") is that existing pages which use lots of it are now damaged and require revision see Wikipedia_talk:Canadian_Wikipedians'_notice_board#Why_are_templates_not_working_on_this_page.3F and template talk:cite bcgnis. The advocate/creator of "cite bcgnis" claimed that coders are told that they shoudl write templates as if there were all teh server space and processor space in the world; but so long as there are limits on the size of pages, and these new template limits, then that's a conundrum. Why can't CONTENT also be written without regard to server/processor space/power? Or is priority being given only to making the INTERFACE as big and bloated as it wants to be, and page-content is to be sacrificed to make room for more code????Skookum1 (talk) 20:15, 14 September 2010 (UTC)

Content has no limits, though page splits are suggested for really long articles. I'm not sure where you got your info about this. Content is generally straight text, or wikilinks and other wiki markup that is usually easy to parse and light on processing load. Templates, on the other hand, may make use of functions which could have recursion or loops, which are processor intensive. Mindmatrix 15:28, 15 September 2010 (UTC)

Added: "Highest expansion depth"

I've added the counter Highest expansion depth as a section, but cannot describe it. Anyone? -DePiep (talk) 10:35, 7 August 2012 (UTC)

Increase post-expand limits

If you encountered

<!-- WARNING: template omitted, post-expand include size too large -->

you can increase $wgMaxArticleSize variable value, this will increase post-expand limits too.

Is there a potential security risk with the NewPP limit report?

At the end of the report it says something like: "Saved in parser cache with key xxxx:pcache:idhas..." etc, where "xxxx" is the database name. Does this represent a potential security risk?

Limit too small for all template documentation cases

See Draft:Template:Hybrid_motor_type/doc. 217.162.112.133 (talk) 22:57, 23 September 2019 (UTC)

So cut it down. Does it really need to show demonstrations of every possible permutation of parameters? --Redrose64 🌹 (talk) 17:54, 24 September 2019 (UTC)

I will not cut it down. They are the test cases for the template. I can check that the code is working properly in the preview before commit. 217.162.112.133 (talk) 18:08, 24 September 2019 (UTC)

The template must be too complex. Given the simplicity of what it does, I don't see why it should exceed the limit. Also: testcases are usually placed under /testcases, not /doc; lint errors must be eliminated if you plan to ever roll it out; I strongly recommend you create an account. Nardog (talk) 18:30, 24 September 2019 (UTC)

Update to WP:Template editor and WP:Template limits

I added this to WP:Template editor#Other considerations and this across several sections of WP:Template limits to encourage editors to pay attention to template limits before changing templates like {{flagicon}} that are used many times on one page, as they can push a page over the limit.

Module:Citation/CS1 and the templates and modules that call it are another example. On pages with hundreds of citations, a seemingly minor change to the citation templates can push a page that has hundreds of citations like Donald Trump (700+ citations) over one or more of the limits. davidwr/(talk)/(contribs) 17:31, 5 March 2020 (UTC)

Discussion on how to deal with PEIS limits at 2019–20 coronavirus pandemic

 You are invited to join the discussion at Talk:2019–20_coronavirus_pandemic#Dealing_with_technical_limitations_of_WP:PEIS. {{u|Sdkb}} ^talk 00:16, 13 April 2020 (UTC)

Analyzing a page's PEIS

If you need to reduce a page's post-expand include size, it would help to understand where that PEIS comes from. So is there a tool that, given a particular page, tells you how much different parts of the page contribute to the PEIS? One possibility would be to break it down by section (so, like the "Section sizes" template, but for PEIS instead of source size), but another would be to break it down by template name. (E.g., all of this page's calls to the "cite" template contribute X to the page's PEIS). Jmdyck (talk) 18:12, 15 June 2020 (UTC)

There’s already a report in the HTML source, which shows template expand times, not PEIS, like this:

<!--
Transclusion expansion time report (%,ms,calls,template)
100.00%   13.596      1 -total
 47.24%    6.423      1 Template:Resolved
 25.82%    3.511      1 Template:Hmbox
 24.92%    3.389      9 Template:Tl
 18.80%    2.556      1 Template:Z48
-->

I can imagine something like this for PEIS as well. (I don’t know how feasible it technically is, though.) —Tacsipacsi (talk) 00:50, 17 June 2020 (UTC)

Necessity of the "post-expand include size"-limit, identifying the core problem and possible alternative solutions

I was running into a problem when trying to transclude multiple sections in something related to the article 2020 in science due to the Post-expand include size limit.

• The problem seems to be that each of the transcluded sections had many references which caused a long Transclusion expansion time (unbelievably over 7 seconds for content that is only text and images which are supposed to lazy-load in 2020)
  • The Transclusion expansion time can be checked in the source code of the article. For a tested version of the article it was (the total % being over 100% seems to be an additional problem):

Transclusion expansion time report (%,ms,calls,template)

100.00% 7733.218 1 -total

61.75% 4774.901 1 Template:Reflist

23.74% 1836.005 293 Template:Cite_journal

22.62% 1749.421 502 Template:Cite_news

5.67% 438.425 4 Template:Fix

5.50% 425.630 1 Template:Overly_detailed_inline

5.47% 422.810 12 Template:Category_handler

4.52% 349.686 1 Template:CVE

3.32% 257.085 80 Template:Cite_web

1.89% 145.880 7 Template:Convert

• Due to these problems I created an issue on phabricator here.
• However my recent questions there have not been answered which may be due to readers of the task not knowing any answer to the questions I posed. This is why I'm asking you, dear watchers/readers of this talk page.
• The problem I'm having seems to be shared with a substantial number of other editors / other articles − including many COVID-19-related ones.
• The questions:
  • Is or can there be any near-term solution for including more text via transclusions (i.e. up to 12 section-transclusions with many references each)?
    • One alternative or near-term solution I could imagine would be a software change that allows for collapsed sections that e.g. only load the data after clicking a [show] button and/or when expanding a section in the mobile view. What do you think of this and is there already an issue for this? It could also detect if the client device is a mobile phone (and/or has a slow Internet-connection) and if not preload the content of these collapsed sections.
  • According to #Why are there limits? the rationale for this artificial limit seems to be that a) the content is considered "large quantity of data" b) the amount is slow to load and c) that it can be "used to mount a denial of service (DoS) attack on the servers".
    • However, 2 MB is not a large quantity of data in 2020 − it's about the size of images commonly shared on image-sharing websites and it should be only a fraction of that size when compressed. Is compression used as much/efficiently as it could be? There are many websites with a) lot more data that b) load a lot quicker and c) often deploy modern Web solutions like lazy-loading content or infinite-scrolling etc.
    • If the Wikimedia servers can't handle requests of 2 MB of uncompressed data (mostly text) I think there's something really wrong with the hard/software, especially when considering the amount of donated money theoretically available to upgrade either or both.
    • I don't think more content in pages would allow for more problematic DoS attacks than otherwise. And if so this should probably be mitigated, just like any other DoS attacks, with adequate technical measures (like limiting the number of requests to a page of large size from a single IP).
  • Are templates and transclusions not prerendered/preparsed after a change to the template or transcluded target section? Does including templates and transclusions always require servers to process every single request? There seems to be something really wrong with the templating architecture if this is the case − no wonder there's a high load on the servers if the templates cause computing-intensive parsing at every page-load. Is there a task and/or suggested solution to this somewhere? From this it seems that transcluded templates are updated after their changed and not somehow dynamically loaded at page-load.
  • Why do the reference-templates take so long to load? Shouldn't they be just text (partly hyperlinks)? There seems to be something really wrong with the reference templating design or template-mechanisms in general if the transclusion expansion time (see the example above) show the real load-times.

I'd appreciate if somebody could answer it here or at the phabricator issue.

tl;dr: just read the text highlighted in bold.

Thank you.

--Prototyperspective (talk) 21:18, 26 September 2020 (UTC)

The limits really do need to be raised, it is 2020. There should also be higher limits on certain "white listed" pages, with the understanding that purging of these pages' caches may be de-prioritized if the work queue gets long. I know you were writing about limits on time but there are whole classes of articles, mostly sports-related, that run up against other limits, including post-expansion include size and expensive function count limits. Wikipedia needs a better way to handle these. davidwr/(talk)/(contribs) 23:43, 26 September 2020 (UTC)

Workaround to the partial-transclusion issue: Instead of marking off sections in article A to be transcluded by Article B, SPLIT Article A into parts, and have both Article A and Article B include the sub-parts. It's not the normal way of doing things, but sometimes when you hit hard limits you have to WP:Ignore all rules to get the job done. Just be sure to document what you did and why you did it so it can be undone when the limits are eventually raised. davidwr/(talk)/(contribs) 23:54, 26 September 2020 (UTC)

Thanks for the helpful comments. I also think they should be raised for at least whitelisted pages. However, I propose to solve the problems that caused people to create these artificial limits (or to have these limits as low as they currently are).

Afaik the "Post-expand include size limit" is directly related to the "Transclusion expansion time" which is why the latter is so long when a page hits the former's limit. I think solving whatever causes long "Transclusion expansion time"s would also allow for higher "Post-expand include size limit"s. From the research, it looks like the reference templates or rather the way templating works in Wikipedia are what cause long "Transclusion expansion time"s.

I'm not sure if I understood your suggested workaround: what I tried was having Article A transclude sections from Article B as well as sections from Article C. So far I haven't seen any temporary workaround in articles which have similar problems, such as COVID-19 pandemic in Japan.

--Prototyperspective (talk) 09:37, 27 September 2020 (UTC)

I don't remember exactly what they are, but I think there are two very large Middle-East-related templates that are each transcluded into more than one article. If limits and performance were not an issue, one or both would be better done as a "transcluded section" in one of the articles which they are currently transcluded into. I do NOT know if they are done up as templates for technical reasons or if that is just an editorial decision made by whoever made them.

Here is something you can try as a test:

Take several science articles and copy them to your personal userspace. Be sure to remove categories and the like.

Carve off sections into stand-alone pages that are in your userspace.

Copy the "big" article that transcludes from several science articles into your userspace, removing categories and the like. Have it transclude these carved-off pages instead.

Compare the parserdata against the parserdata for the "live" version and see if there is any significant difference.

davidwr/(talk)/(contribs) 15:55, 27 September 2020 (UTC)

Move to help namespace?

Looks like a help page to me — Martin (MSGJ · talk) 20:20, 21 October 2020 (UTC)

Constant expressions

Will constrant expressions only be expanded once (until the template code is modified)? Trigenibinion (talk) 18:22, 22 January 2021 (UTC)

Whitespace

It is very bad that whitespace is being counted. Trigenibinion (talk) 14:05, 27 January 2021 (UTC)

Discussion at Wikipedia:Categories for discussion/Log/2021 May 7 § Category:Pages where template include size is exceeded

 You are invited to join the discussion at Wikipedia:Categories for discussion/Log/2021 May 7 § Category:Pages where template include size is exceeded. * Pppery * _{it has begun...} 17:12, 7 May 2021 (UTC)

Requested move 18 March 2022

The following is a closed discussion of a requested move. Please do not modify it. Subsequent comments should be made in a new section on the talk page. Editors desiring to contest the closing decision should consider a move review after discussing it on the closer's talk page. No further edits should be made to this discussion.

The result of the move request was: moved. (closed by non-admin page mover) Extraordinary Writ (talk) 21:19, 25 March 2022 (UTC)

---

Wikipedia:Template limits → Help:Template limits – Appears to be a help page. Pinging MSGJ, who originally proposed this. 🐶 EpicPupper ^{(he/him | talk)} 05:02, 18 March 2022 (UTC)

• Happy to support — Martin (MSGJ · talk) 12:31, 18 March 2022 (UTC)
• Support, just a plain help page really. --Jules (Mrjulesd) 12:58, 23 March 2022 (UTC)

The discussion above is closed. Please do not modify it. Subsequent comments should be made on the appropriate discussion page. No further edits should be made to this discussion.

Workaround for #ifexist expensive parser function

Users interested in WP:EXPENSIVE parser functions, especially #ifexist, may be interested in the discussion WP:VPT#PAGESIZE as inexpensive alternative to #ifexist parser function. Thanks, Mathglot (talk) 03:45, 15 March 2024 (UTC)