Restricting the software that runs on a computer to that signed by some authority does absolutely nothing to ensure that the software is secure (free from bugs that compromise your data, or from bugs that enable your machine to be used by malicious third parties). The Apple App Store paradigm is about monopolistic control of a commerce distribution channel; it cannot do anything about software security or quality (and might even impact the latter negatively).
What about virtualization? One promise is that applications running in a virtual machine can be restricted to access just a relevant subset of your data. Even assuming a perfectly secure, bug-free, and siloed virtualization scheme, you cannot tell whether the app that the virtual machine will run does not contain secret back-doors or information leaking channels (both can be accomplished using steganography to hide things in the data the app consumes and produces as part of its regular operation). Checking compiled binaries for these is simply not feasible.
The only alternative that is left is the one advocated by Richard Stallman - you cannot trust a program unless you can read its source code. Hiding back doors in source code is more difficult to do, easier to spot, and, with version control systems, possible to track and attribute.
While there is no way to prevent these two techniques (and in fact it is undesirable to do so), you certainly should be able to have the freedom to use applications written and audited by people you trust. The centralized, signed app store approach used by Apple destroys this continuum of trust by putting all applications on an equal level. "Approved by Apple" does not mean much when the approval process is secretive, arbitrary, and does not guarantee quality or security.
One way to encourage the ideal of "easily auditable source" is with Free Software licensing. This is where the innovation of Henry Poole comes in handy - the Affero GPL is the most business-friendly of all Free Software licenses.
Are there other benefits to source code-only distribution beside security? Plenty: complete portability, high performance (the ability to compile to native code), tiny download sizes, and easy dependency management.