Talk:LTI system theory

From Wikipedia, the free encyclopedia
Jump to: navigation, search
          This article is of interest to the following WikiProjects:
WikiProject Time (Rated C-class, Low-importance)
WikiProject icon This article is within the scope of WikiProject Time, a collaborative effort to improve the coverage of Time on Wikipedia. If you would like to participate, please visit the project page, where you can join the discussion and see a list of open tasks.
C-Class article C  This article has been rated as C-Class on the project's quality scale.
 Low  This article has been rated as Low-importance on the project's importance scale.
 
WikiProject Systems (Rated C-class, High-importance)
WikiProject icon This article is within the scope of WikiProject Systems, which collaborates on articles related to systems and systems science.
C-Class article C  This article has been rated as C-Class on the project's quality scale.
 High  This article has been rated as High-importance on the project's importance scale.
Taskforce icon
This article is within the field of Control theory.
 


Example[edit]

Need to put an example for the discrete time case to make things a little clearer.

  • I say this side focuses too much on continuous time (CT). The concept of LTI should be explained independent of whether or not the time is continuous. For example, it is difficult to link from a side which discusses digital signal processing algorithms to this LTI side, because it only talks about CT signal processing. Also, LSI (Linear Shift Invariance), which means basically the same thing, must be mentioned. Faust o 20:25, 23 January 2006 (UTC)

i think there is a lot more that can be done to make this clearer.[edit]

but i'm glad the article is there. this should be written so that it can be understood by someone who doesn't already understand it. to begin with, i think there needs to be a better introduction as to what \mathbb{H} is. what are the fundamental properties of this LTI operator \mathbb{H}? first exactly what does it mean for \mathbb{H} \{ x_1(t) + x_2(t) \} to be linear (the additive superposition property) and then what does it mean for \mathbb{H} \{ x(t-\tau) \} to be time-invariant. then from that derive the more general superposition property, then introduce the dirac delta impulse as an input and define the output of \mathbb{H} \{ \delta(t) \} to be the impulse response. since the article is LTI, there is no need to introduce h(t_1,t_2). all that does is obfuscate.

Mark, i hope you don't mind if i whack at this a bit in the near future. i gotta figure out how to draw a png image and upload it. r b-j 04:53, 28 Apr 2005 (UTC)

I disagree. I think h(t_1,t_2) should be there, as it shows how much simpler things become when translation invariance is imposed. I think you can represent any linear system with \int_{-\infty}^\infty h(t_1,t_2) x(t_2) d t_2, but I'm not 100% sure. Certainly any nice system. - Jpkotta 06:40, 12 February 2006 (UTC)
you're correct when you say that "you can represent any linear system with \int_{-\infty}^\infty h(t_1,t_2) x(t_2) d t_2", but that is more general than LTI and unnecessarily complicated. that expression includes Linear, time-variant systems also. perhaps the article should be just "Linear systems" and deal with both time-variant and time-invariant. r b-j 07:25, 14 February 2006 (UTC)


Representation[edit]

It appears that it is assumed that the LTI linear system can be represented by a convolution. However, in Zemanian's book on distributions a result due to Schwatrz and its proof are presented. The result has to do with sufficient conditions under which an LTI transformation can be represented by a convolution. I guess that continuity of the LTI transformation is one of the conditions. The result appears to be quite deep. Some other proofs in the literature may not be real proofs. This result is not as simple as one might think.

Yaacov

I deleted the word "integral" from my comment. The convolution of didtributions has a definition that does not appear to rely on integration.

Yaacov

I added: "Some other proofs in the literature may not be real proofs. This result is not as simple as one might think."

Yaacov

Notation[edit]

Is the bb font  \mathbb{H} at all common for an operator? I've never seen that before, and I think bb font should be reserved for sets like the reals and complexes. A clumsy but informative notation that one of my professors uses is this:
 \mathcal{L}_{t_o}\{x(t_i)\}_{t_i} ,
where \mathcal{L} is the operator, to is the output variable, and ti is the input variable. To say that a system is TI,
 \mathcal{L}_{t}\{x(t + \tau)\}_{t} = \mathcal{L}_{t+\tau}\{x(t)\}_{t}.
I'm not sure if it's a good idea to use it here though... -- Jpkotta 06:49, 12 February 2006 (UTC)

Discete time[edit]

Should discrete time be folded in with continuous time, or should there be two halves of the article?

By folded, I mean:

  • basics
    • what it means to be LTI in C.T.
    • what it means to be LTI in D.T.
  • transforms
    • laplace
    • z

By two halves, I mean

  • C.T.
    • what it means to be LTI
    • Laplace
  • D.T.
    • what it means to be LTI
    • z transform

I vote for the two halves option, because then it would be easier to split into two articles in the future. -- Jpkotta 06:46, 12 February 2006 (UTC)

I made a big update to the article, and most of it was to add a "mirror image" of the CT stuff for DT. There is a bit more to go, but it's almost done. -- Jpkotta 22:25, 21 April 2006 (UTC)

Comparison with Green function[edit]

This page has the equation

y(t_1) = \int_{-\infty}^{\infty} h(t_1, t_2) \, x(t_2) \, d t_2

which looks an awful lot like the application of a Green function

 u(x) = \int_0^\ell f(s) g(x,s) \, ds

however this page doesn't even mention Green functions. Can someone explan when the two approaches can be applied? (My hunch right now is that Green functions can be used for linear systems that are not necessarily time-invariant.) —Ben FrantzDale 03:24, 17 November 2006 (UTC)

Yes, a Green's function is essentially an impulse response. Different fields have developed different terms for these things. But the Green's function is also more general, as you note, than is needed for time-invariant systems; as is that h(t1,t2) integral. Dicklyon 06:14, 17 November 2006 (UTC)

Discrete example confusion[edit]

The first example starts out describing the delay operator then describes the difference operator. 203.173.167.211 23:03, 3 February 2007 (UTC)

I fixed it. And changed z to inverse z for the delay operator. I'm not sure where that came from or whether I've left some discrepancy. Dicklyon 01:30, 4 February 2007 (UTC)

Problem with linearity[edit]

Generally, it is not true that for a linear operator L (such that L(ax_1 + bx_2) = aL(x_1) + bL(x_2)), that

L\left(\sum_n a_nx_n \right) = \sum_n a_n L(x_n)

over arbitrary index sets (i.e. infinite sums). This is used heavily in LTI analysis.

The result does not follow from induction. So why should it be true for linear systems? I think linearity itself is not strong enough a condition to warrant the infinite-sum result. Are there deeper maths behind systems analysis that provide this result? (For example, restriction of linear systems to duals of certain maps is a sufficiently strong condition to imply this result.) 18.243.2.126 (talk) 01:42, 13 February 2008 (UTC)

Are you saying it's not true? Or that you don't know how to prove it? Do you have a counter-example? Dicklyon (talk) 05:07, 13 February 2008 (UTC)
It is plainly not true, if linearity is the only condition being imposed (the constant signal x[n] = 1 is linearly independent from the unit impulses and all their shifts -- while this is not the case if you allow infinite sums); I have yet to construct a viable time-invariant counterexample; time-invariant functionals tend to be a lot more restrictive. For example, any time invariant system which only outputs constant signals is identically the zero system. 18.243.2.126 (talk) 00:21, 19 February 2008 (UTC)
I'm not following you. What is the concept of "linearly independent" and how does it relate to the question at hand? Dicklyon (talk) 06:10, 19 February 2008 (UTC)
The terminology comes from linear algebra (the wiki article explains it better than I can in a short paragraph). Note that the set of real-valued signals forms a real vector space. 18.243.2.126 (talk) 19:28, 20 February 2008 (UTC)
I understand about linear algebra and vector spaces, but there's nothing in this article, nor in linear algebra about this concept you've brought up, so tell us why you think it's relevant. Dicklyon (talk) 20:05, 20 February 2008 (UTC)
I see that linear independence says "In linear algebra, a family of vectors is linearly independent if none of them can be written as a linear combination of finitely many other vectors in the collection." This renders your above statement "while this is not the case if you allow infinite sums" somewhat meaningless. So I still don't get your point. Dicklyon (talk) 20:10, 20 February 2008 (UTC)
In what way does it "not follow from induction"? Oli Filth(talk) 12:02, 13 February 2008 (UTC)
Induction can prove it for all finite subsequences of the infinite index sequences, without bound, but not for the infinite sequence itself. Dicklyon (talk) 16:24, 13 February 2008 (UTC)

New footnote[edit]

I've removed the recently-added footnote, because I'm not sure what it says is relevant. The explanation of the delay operator is purely an example of "it's easier to write", as by substitution, z = e^{sT}. The differentiation explanation is irrelevant, because when using z, we're in discrete time, and so would never differentiate w.r.t. continuous time. Oli Filth(talk) 21:28, 10 April 2008 (UTC)

"any input"?[edit]

Is this nonsense?:

"Again using the sifting property of the \delta(t), we can write any input as a superposition of deltas:
x(t) = \int_{-\infty}^\infty x(\tau) \delta(t-\tau) \,d\tau "

--Bob K (talk) 00:52, 11 June 2008 (UTC)

Other than special cases which one would need to resort to Lebesgue measures to describe, I don't think it's nonsense. What specifically are you questioning, the maths itself, or the use of the qualifier "any"? Oli Filth(talk) 08:05, 11 June 2008 (UTC)


It sounds like we are saying that delta functions are a basis set (like sinusoids) for representing signals.
Anyhow, I think the "proof" was redundant at best. At worst, it was circular logic, because it uses this formula:
 (h * \delta) (t)  = \int_{-\infty}^{\infty} h(t - \tau) \, \delta (\tau) \, d \tau = h(t),
which is a special case of this one:

y(t_1) = \int_{-\infty}^{\infty} h(t_1 - t_2) \, x(t_2) \, d t_2 \,

 

 

 

 

(Eq.1)

to derive this one:
\mathcal{H} x(t) = \int_{-\infty}^\infty x(\tau) h(t-\tau) \,d\tau
which is the same as Eq.1.
--Bob K (talk) 13:24, 11 June 2008 (UTC)
Just for the record, I think what someone was trying to do is done correctly (but for the discrete case) at Impulse response.
--Bob K (talk) 19:21, 13 July 2008 (UTC)
In the discrete case, the delayed delta signals certainly are a basis set. I'm not sure about the continuous case. However, I agree that the proof was somewhat circular. Oli Filth(talk) 15:45, 11 June 2008 (UTC)
Well besides that fact that neither of us has ever heard of using Dirac deltas as a basis set for all continuous signals, how does one make the leap from that dubious statement to this?:
x(t) = \int_{-\infty}^\infty x(\tau) \delta(t-\tau) \,d\tau
I believe it is just nonsense.
--Bob K (talk) 18:04, 11 June 2008 (UTC)
We know the following is true (it's the response of system x(t) to an impulse):
\ x(t) = \int_{-\infty}^\infty \delta(\tau) x(t - \tau) \,d\tau
and we also know that convolution is commutative. Oli Filth(talk) 18:24, 11 June 2008 (UTC)
Those two statements of yours have nothing to do with the assertion that Dirac deltas are a basis set for all continuous signals and therefore nothing to do with my question.
--Bob K (talk) 23:24, 11 June 2008 (UTC)
You said "besides that fact" above. I'm not defending the material (I didn't write it); just playing devil's advocate... I feel perhaps we are talking at cross purposes! Oli Filth(talk) 23:30, 11 June 2008 (UTC)
Out of context, the integral formula is fine. But I quoted the entire statement and asked if it makes any sense. (You said it does.) Even if you accept the idea of Dirac deltas as basis functions [I don't], how does the integral formula follow from that? The whole thing just seems silly.
--Bob K (talk) 00:17, 12 June 2008 (UTC)
There is no circular logic, all it uses is the sifting property.
Consider the identity system, i.e. the system that simply returns the input, unchanged. It is clearly LTI. What is the impulse response of the identity system? How would you compute the output using a convolution? This leads directly to the above statement.
If you accept that sinusoids are basis functions, then why can't deltas be basis functions? The Fourier transforms of the sinusoids are deltas. I can take inner products of a signal and deltas (another way to interpret the above formula) and completely specify the signal up to a set of measure zero, just as I can with sinusoids.
Jpkotta (talk) 03:51, 6 October 2008 (UTC)

Is Overview written at the right level?[edit]

In the Overview section, this paragraph abruptly appears:

For all LTI systems, the eigenfunctions, and the basis functions of the transforms, are complex exponentials. This is, if the input to a system is the complex waveform A\exp({st}) for some complex amplitude A and complex frequency s, the output will be some complex constant times the input, say B\exp({st}) for some new complex amplitude B. The ratio B/A is the transfer function at frequency s.

But complex amplitude and complex frequency are not really the inputs and outputs of the system. It's like saying that radios transmit and receive analytic signals. For the sake of those who don't already know the subject, wouldn't it be better to stick closer to reality than to mathematical abstractions?

--Bob K (talk) 11:32, 18 June 2008 (UTC)

For practical (i.e. physically-realisable) systems using actual physical quantities as the "signals", then yes, it's true that the signals will be real. But I would have said that this article is about the mathematical abstraction, i.e. LTI systems in their most general form, not the subset of the theory that covers real-only signals and systems. Therefore, the complete set of eigenfunctions really is the complex exponentials. I don't think we can avoid discussing this. We could add the caveat that for real-only signals, the eigenfunctions are of the form \exp(\alpha t) . \cos(\omega t + \phi) (for real \alpha, \omega and \phi), or we could explicitly explain what is meant by "complex frequency", etc. Oli Filth(talk) 12:06, 18 June 2008 (UTC)
Those aren't eigenfunctions, since the phase can change in going through the system. You really do need complex exponentials, not just different phases of damped cosines. Dicklyon (talk) 16:38, 30 January 2009 (UTC)
Imaginary functions do exist in real life. Every time you generate a cosine, you're actually generating two imaginary functions and summing them. By the linearity property, we handle the analysis for the two terms separately. That's not only the simplest explanation, but it's the most accurate. The imaginary analysis is not simply an abstraction; it's a useful decomposition of reality. (On a different note, it's useful to think about transmission of purely imaginary signals when describing single-sideband modulation as the baseband signal is made asymmetric after post-modulation filtering)TedPavlic (talk) 16:26, 30 January 2009 (UTC)
You don't think it's fair to admit that the decomposition of a cosine into a pair of complex exponentials is a decomposition of a "real" (e.g. voltage) signal into a pair of "abstract" or non-real mathematical functions? These are no longer things that can be independently sent on a wire, for example. We use them because they are the eigenfunction of continuous-time linear systems; that doesn't make them "real". Even SSB signals are real functions of time, not complex signals. In real life, all signals are real-valued, except when we use a pair of real signals, or a modulated real signal, to "represent" an otherwise non-real signal. Dicklyon (talk) 16:35, 30 January 2009 (UTC)
It's fair, but I just think it's unnecessarily complex (pun intended? not sure) to treat the complex exponentials as abstract magic. They come right out of the definition of a (damped) sinusoids. There is a straightforward jump from real signals to decompositions of complex exponentials, and I favor that straightforward leap rather than something touchy feeley. —TedPavlic (talk) 17:15, 30 January 2009 (UTC)
I wouldn't advocate anything touchy-feely either. If eigenfunctions are introduced, complex exponentials are unavoidable. And they're so useful that we shouldn't avoid introducing them. We need to just note that real inputs and outputs decompose into pairs of these eigenfunctions. For the intro, however, we might be able to defer that discussion and talk about linearity and time invariance without eigenfunctions, no? Dicklyon (talk) 18:46, 30 January 2009 (UTC)
I agree. With my young students (sophomore-level engineers), I take the approach in this short document. In two pages, I try to cover all of the math they'll need to go from first principles to LTI stability, and I didn't introduce the term eigenvalue until well-into the stability discussion on the second page (and I don't give the formal definition; I justify its name using stability). The utility of having eigenvalues and eigenfunctions is obvious when just working the systems out. Naming them initially just confuses the matter (I think). Likewise, I think this article need not get deep into the abstraction. It can:
  1. Define linear systems (scaled sums)
  2. Use the definition to motivate using complex exponential functions
  3. Show how real signals can be composed from complex exponentials
The order of 2 and 3 is unclear to me. However, I don't see a reason to really get deep into the algebra until at the end (or not at all?). Technically you need it in order to show that you can eigen-decompose real functions, but I think it's OK to push some of the justification for that off into some of the transform pages. —TedPavlic (talk) 19:30, 30 January 2009 (UTC)
That's a pretty nice develoment. But where you say "Consider functions of the form...," you do slip into complex signals in a context where previously you were talking about real-world variables. If at that point you explain that real signals can be decomposed into such things, it would be fine, but as it stands, that gap in the logic may confuse some students. Same problem a bit later when you say "We will assume that this output is made up of complex exponentials...;" the student who expects a real output may not know what to make of this. Dicklyon (talk) 04:19, 31 January 2009 (UTC)

The next paragraph after the one I quoted does mention that real signals are a subset. Perhaps it could be said in a more accessible way to more readers, but it's much better than nothing. This probably isn't the right place to explain complex frequency, but it would be an improvement if complex frequency was an internal link to an understandable article on that subject. Do we have such an article?

--Bob K (talk) 14:12, 18 June 2008 (UTC)

BTW, the article Phasor (sine waves) is an example of a more accessible approach, in my opinion, because it puts the complex representation into a context that more people can relate to. It motivates the introduction of complex amplitudes by using them to reduce a "real" problem to an elegant, easily solvable, equation:

j \omega V_c + \frac{1}{RC} V_c = \frac{1}{RC}V_s.

And then it shows the additional steps to extract the "real" solution from the complex result. So from that perspective, the concept of complex amplitude is just an intermediate and temporary step in a longer process. It's actually the subset, not the superset. It is only one of the tools neded to understand all LTI systems (i.e., including realizable ones).

--Bob K (talk) 15:18, 18 June 2008 (UTC)

But that's backwards. There are several ways to show that 2 cos(\omega t) = \exp(j\omega t)+\exp(-j\omega t), and most people probably recall something like that from high school (or lower?). By the definition of an LTI system, whenever the input is the sum of two signals, we can treat each signal independently and sum the result. So we handle \exp(j\omega t) and then \exp(-j\omega t) and then sum the result. It's misleading to say that we're taking the "real part" of anything here. We're not throwing anything away. We're using both the real and imaginary parts. It just so happens that since we started with all real signals, we end with all real signals. (note that there are plenty of applications where you start with complex signals and thus need complex signals out, and so it's best to leave the description as general as possible)TedPavlic (talk) 16:32, 30 January 2009 (UTC)

math error[edit]

In the section LTI_system_theory#Time_invariance_and_linear_transformation we say:

y(t_1) = \int_{-\infty}^{\infty} h(t_1, t_2) \, x(t_2) \, d t_2.

If the linear operator \mathcal{H} is also time-invariant, then

 h(t_1, t_2) = h(t_1 + \tau, t_2 + \tau)  \qquad  \forall \, \tau \in \mathbb{R}.

For the choice

 \tau = -t_2, \,

it follows that

h(t_1, t_2) = h(t_1 - t_2, 0).  \,

But we can't make that choice, because t_2\, is the variable of integration, and \tau\, is a constant time offset.

The "proof" has been fudged so as to time-reverse the weighting function (as it was defined above) so that it looks like an impulse response. But an impulse response is a weighting function whose value at \tau\, is the weight applied to the value of the input at time t-\tau\,, where t\, is the time of the desired output value. That means it is a time-reversed version of our definition of h(t_1, \tau).\,

So one way to fix the problem would be to start with the definition:

y(t) = \int_{-\infty}^{\infty} h(t, -\tau) \, x(\tau) \, d \tau.

 

 

 

 

(Eq.2)

But the -\tau\, will probably confuse people. So an alternative is to show that if h\, is defined by:

y(t) = \int_{-\infty}^{\infty} h(t, \tau) \, x(\tau) \, d \tau,

the impulse response is h(0,-\tau).\, Then we could point out that defining h(0,\tau)\, as an impulse response leads to Eq.2 (and vice versa).

--Bob K (talk) 03:16, 19 June 2008 (UTC)

another math error[edit]

In section LTI_system_theory#Time_invariance_and_linear_transformation we say:

y[n_1] = \sum_{n_2=-\infty}^{\infty} h[n_1, n_2] \, x[n_2],

If the linear operator \mathcal{H} is also time-invariant, then

 h[n_1, n_2] = h[n_1 + m, n_2 + m]  \qquad  \forall \, m \in \mathbb{Z}.

If we let

 m = -n_2, \,

then it follows that

h[n_1, n_2] = h[n_1 - n_2, 0]. \,

But we can't "let m = -n_2,\," because n_2\, is the variable of integration, and m\, is a constant offset.

--Bob K (talk) 14:06, 19 June 2008 (UTC)

Scaling from superposition[edit]

I read: "It can be shown that, given this superposition property, the scaling property follows for any rational scalar." Please correct me if I am wrong (I am not an expert), but I think it would be better to write "given this superposition property, the scaling property obviously follows for any rational scalar" (or any equivalent wording). Indeed, it seems to me that (using notations from the superposition property explanation) we simply have to take c2=0 to get the scaling property. The current wording let the reader thinks that the proof is not obvious, IMHO.--OlivierMiR (talk) 13:10, 27 January 2009 (UTC)

The scaling property does not follow from simple additivity. For more information, see linear map. You need both additivity and homogeneity (of order 1) to call something linear. That is,
f(a_1 x_1+\cdots+a_m x_m)=a_1 f(x_1)+\cdots+a_m f(x_m)
This statement combines the two properties. It's a concise statement of two independent requirements. Note that functions can be created that are additive but not linear. —TedPavlic (talk) 16:18, 30 January 2009 (UTC)

LTI.png shouldn't say "Frequency domain"[edit]

The included LTI.png image:

LTI.png

shows a transformation from "Time domain" to "Frequency domain." However, clearly the s (or "Laplace") domain is shown, which is a generalization of the frequency domain. Either each s should be changed to a j\omega or the "Frequency domain" should be changed to something like "Laplace domain" or "s domain." —TedPavlic (talk) 22:55, 28 January 2009 (UTC)

It's not so bad. The Laplace domain (complex frequencies) is probably the most commonly used frequency domain in this application. Dicklyon (talk) 00:07, 29 January 2009 (UTC)

Zero state response[edit]

Zero state response also discusses the analysis of linear systems, but does not make a specific restriction of the problems to time-invariant system. There is, as yet, no top-level article on Linear system theory that deals with both this and the more general case of time-variant linear systems. Is there a possible route for refactoring/merging this material? -- The Anome (talk) 02:54, 17 February 2010 (UTC)

Dubious (and misleading!) example: "ideal LPF not stable"[edit]

This paragraph:

As an example, the ideal low-pass filter with impulse response equal to a sinc function is not BIBO stable, because the sinc function does not have a finite L1 norm. Thus, for some bounded input, the output of the ideal low-pass filter is unbounded. In particular, if the input is zero for t < 0\, and equal to a sinusoid at the cut-off frequency for t > 0\,, then the output will be unbounded for all times other than the zero crossings.

It is really hard to imagine how a filter with a perfect LPF frequency response (as a sinc filter has) could be classified as unstable. Any sinusoidal input, for instance, is a bounded input with an obviously bounded output (either the same sinusoid or zero, depending on its frequency). However it indeed appears that the L1 criterion mentioned here would be violated by the sinc impulse response, which is stated to be an absolute test for stability/instability.

But never mind: I believe I see the problem. The sinc function extends to infinity in both negative and positive time so it cannot possibly be implemented as a causal filter, and this section is about causal filters (otherwise the concept of instability breaks down inasmuch as stable impulse responses with right hand plane zeros, if reversed in time, describe unstable systems). So I don't think the example is applicable.

And in any case, it could only be confusing to an average WP reader who is trying to LEARN about systems (since it's confusing to ME and I sort of thought I knew all about filter theory!). If the claim is true in some sense, then it stands more as a paradox or riddle than as useful information. Could someone remove this and put in a better example? And possibly (but here I'm not certain) restate the L1 criterion with a statement that it only applies to causal systems, or whatever qualifications are missing which make this result paradoxical or (I think) simply wrong? Interferometrist (talk) 12:14, 3 March 2011 (UTC)

You are correct in saying "It is really hard to imagine how a filter with a perfect LPF frequency response (as a sinc filter has) could be classified as unstable." It is not unstable; neither is its time reversal. But it is also not bounded-input–bounded-output stable. So what? It is still a good illustration of the idea of BIBO stability. Dicklyon (talk) 06:17, 7 March 2011 (UTC)