When the Torah discusses the Bigdei Kehunah, the priestly garments, it says to make a Tzitz made of thin pure gold and to engrave in it the words "Kodesh LaHashem" (שמות כ"ח:ל"ו). The Posuk then says that the Kohein Gadol should keep this on his forehead at all times (שם פסוק ל"ח), which Chazal understand to mean that the Kohein Gadol must always be aware of the presence of the Tzitz. The Gemara in Yoma (דף ז:-ח.) says that one is obligated to touch his Tefillin every so often because of a Kal VaChomer derived from the Tzitz. The Tzitz has only one mention of Hashem's name, and nevertheless the Torah says that it must always be on the Kohein Gadol's forehead which means that he should never be מסיח דעת , remove his thoughts, from it; Tefillin which has many mentions of Hashem's name should certainly be the same. Rashi (לשבת דף י"ב. בד"ה והיה) notes that it already had said earlier in the Posuk that the Tzitz must be on Aharon's forehead so the location, the forehead, is already known and thus the later phrase is available for this Drasha.
The Torah Temimah quotes Tosafos in Yoma )דף ח: בד"ה ומה( as saying that this Kal VaChomer is not good because the Tzitz is different since the name of Hashem is revealed and visible on it, as opposed to Tefillin where it is covered. Tosafos therefore holds that this rule of not removing one's attention from Tefillin is only MideRabbanan. The Torah Temimah then says that many argue on this and feel that the rule is MideOraisa, because it is indeed a full fledged Kal VaChomer since there is no practical difference hether Hashem's name is revealed or concealed.
The Torah Temimah himself though, agrees with Tosafos because there are in fact a number of stringencies which apply to the Tzitz and not to Tefillin, presumably because Hashem's name is on the outside. He cites an example of this from a Gemara in Kiddushin (דף ס"ו.) when Yanai HaMelech was told to stand in honor of the Tzitz, a requirement not found by Tefillin. Another example is from a Gemara in Sotah (דף ל"ח.) which says that when the Kohein Gadol Duchens, he does not raise his hands higher than the Tzitz, unlike all the other Kohanim who raise them above their heads (even when wearing Tefillin). Many Rishonim such as the Rosh and the Tur agree with Tosafos and hold that the real issue to be concerned about when wearing Tefillin is not היסח הדעת, but קלות ראש, inappropriate light-headedness. Tosafos admits, though, that even though the Kal VaChomer is invalid, there is value to periodically touching one's Tefillin to remind one's self that they're on.
In any case, no matter which view makes more sense, and whether this rule is from the Torah or from the Rabbanan, the underlying important message is that when performing a Mitzvah, one should not simply do it perfunctorily, but rather, one should always be cognizant of and have an understanding of why and how he is doing the Mitzvah. One should touch the Tefillin because one must remember that they're on, and be aware of what he is doing, just as the Kohein Gadol always had to be aware of the Tzitz. This type of understanding should hopefully heighten our performance of our Mitzvos and our awareness of our relationship with and our obligations to Hashem.