Forums

Favorite VM for development?

Started by Dave Nadler July 12, 2014
On 13/07/14 06:38, Don Y wrote:
> Hi Dave, > > On 7/12/2014 11:20 AM, Dave Nadler wrote: >> So here I am, trying to (again) help (another) customer whose >> previous consultant didn't leave complete sources and tools, and who >> cannot reproduce their production firmware. Aarrgggg... > > <grin> Amazing how often this happens! Even folks missing > *sources*! (oops!)
Or, more subtly, not the /real/ sources. I've seen "source code" that was incomprehensible spaghetti C. Clearly the contractor had developed the FSMs in a development tool that emitted C which was then compiled by a standard toolchain. Perfectly good way of developing a system, but not, IMNSHO, an acceptable way of delivering it. Good for job security, though.
Hi Tom,

On 7/12/2014 11:30 PM, Tom Gardner wrote:
> On 13/07/14 06:38, Don Y wrote: >> On 7/12/2014 11:20 AM, Dave Nadler wrote: >>> So here I am, trying to (again) help (another) customer whose >>> previous consultant didn't leave complete sources and tools, and who >>> cannot reproduce their production firmware. Aarrgggg... >> >> <grin> Amazing how often this happens! Even folks missing >> *sources*! (oops!) > > Or, more subtly, not the /real/ sources.
(sigh) No, I've had projects where I was given product from the factory with *ROMs* and told to make a particular set of changes. "Um, where's the source code for these ROMs? (and the other programmable logic in the design that would enable me to figure out what thhardware actually *does*)" "We lost it" "Oh. Then, I can only give you a T&M contract..."
> I've seen "source code" that was incomprehensible spaghetti C. > Clearly the contractor had developed the FSMs in a development > tool that emitted C which was then compiled by a standard > toolchain. > > Perfectly good way of developing a system, but not, IMNSHO, > an acceptable way of delivering it. Good for job security, > though.
Deliverables should include everything that the client needs to "pick up where you left off". If you "needed" a particular tool (hardware or software) to create your "key deliverable" (e.g., a binary for a piece of code), then that tool should be included (i.e., if it is something you wrote) or otherwise obtainable independently by the client (e.g., let him buy his own license for COTS tools). [If I've created a tool, I am not obligated to provide the *source* for that tool -- unless it is specifically covered in the contract -- just a binary that the client can run to reproduce the items I've delivered] However, I am free to throw together "informal" tools that are not included in the deliverables if those tools existed only to make me more efficient with my time. For example, I build "screens" with a little kludge I wrote many years ago (WYSIWYG interface). But, the output of this tool is something that another developer could create and manipulate directly -- just not as *effectively* as I did. OTOH, things like the rule compiler for the TTS system is effectively "essential" for building accurate rules (you could "hand compile" them but you would spend a LOT more time debugging your mistakes. So, the compiler is included in the deliverables. But, the *source* for the compiler need not be. As long as I am willing to stand behind the rule compiler's accuracy and reliability (i.e., ensure that it does what *I* have advertised it to do; nothing more!)
On 12/07/14 20:20, Dave Nadler wrote:
> So here I am, trying to (again) help (another) customer whose previous consultant didn't leave complete sources and tools, and who cannot reproduce their production firmware. Aarrgggg... > > In past, we've required customers to install the tools and do the builds from our delivered sources, but that is time-consuming for them and doesn't yield repeatable results for certain toolchain installations that wander off and fetch mysterious components from the net (and make it impossible to back up an installation snapshot). > > So, to avoid further repeats, I'm going to do development in a (windows 7 32-bit) VM with EVERYTHING required. Deliveries will be copies of the VM and I will ensure the customer actually rebuilds their production releases in the delivered VM. No more confusion about which version of tools, libraries, sources, etc.... > > I've avoided doing this in past because of VM problems: > - USB bugginess and/or performance interfering with JTAG dongles, and > - VM performance > - annoying 100GB delivery files > > What VMs have you guys used successfully? > The tools for this customer's projects include: > - Microchip MPLABX with ICD-3 debug adapter > - TI CCS for MSP430 (GCC version) using TI MSP-FET430UIF > - (phasing out) Rowley Crossworks for MSP430 using TI adapter > > Host for the VM will be Win7 32-bit, and later Linux and Win7 64-bit. > > I've used QEMU in past for developing inside a Linux VM, and VMware player for a WinXP machine. I tried VirtualBox but was stymied by USB issues. But all of these were at least a couple years ago... > > Any advice, suggestions, warnings much appreciated! > Thanks, > Best Regards, Dave >
Two points: I have found VirtualBox to be excellent. It does everything you need here - USB works fine, and I've run several JTAG debuggers from within VBox machines. It is free (most of it is open-source "free", the USB bit is merely cost-free), which means you don't have to worry about licenses or other complications when installing it everywhere. And the dynamic VDI disk images are reasonably compact - though image files will still be big. It works fine on a Windows host, though Linux makes a better host for any kind of VM. Secondly, try /very/ hard to make your virtual machine Linux rather than Windows. Windows licensing gets complicated when you start copying the VM or transferring it to a third party. If your customer worries about the details of such things (many don't), it will definitely be a problem. And if you try to use the same VM on different machines at the same time, with each one faffing around with updates and "calling home" to MS, you'll find your VM declaring you a pirate and de-activating itself. And if this is for long-term projects, you might find customers complaining that they can't use a Win7 machine because it is "insecure" - their IT folk will only let them run Win9 or Win10 (or whatever it is called in the future). With a Linux VM, you pretty much avoid all this. In a similar vain, try to make sure that none of the tools inside the VM require licenses, node locks, or anything of that sort. CCS with gcc should be fine, but I don't know anything about MPLABX.
On 12.7.14 23:03, Theo Markettos wrote:
> Les Cargill <lcargill99@comcast.com> wrote: >> USB support seems okay - with the exception that I always >> have to do a forced remount ( -o force ) of spinning USB drives on an >> old SuSE 11.0 VM. I've no way of knowing right now if that's an >> artifact of SuSE, VirtualBox or the Win8 host. It's hardly a >> show-stopper and it may actually be fixable. >> >> The VM files themselves are considerably less than 100GB in my case. >> >> This being said, I'm not trying to connect a JTAG adapter. >> That's probably the main risk here. > > I've done USB JTAG things on VMWare Fusion: Mac OS host, Win 7 guest. It > works fine, with the exception of USB3 devices which VMWare doesn't have a > Windows guest driver for. Putting a USB2 hub between the device and the > machine should fix it. > > It's quite neat in that, whenever you plug in a USB gadget, it asks whether > you want to connect it to Mac or choose a VM to connect to (you can of > course remember this setting). > > Theo
Another vote for VMware Fusion. It works fine with several different guests. I'm primarily using different flavors of Linux, but BSD and Solaris work also (And Windowses, if you have to). The guest images work fine when moved to other computers with VMware or VMware Player. Another nice property is that you can use a CD/DVD image on the host computer and show it as a CDS/DVD drive with disk to the guest, so there's no need to burn installation frisbees. -- Tauno Voipio
On 13.7.14 07:11, Robert Wessel wrote:
> On Sun, 13 Jul 2014 02:01:08 +0100, Paul E Bennett > <Paul_E.Bennett@topmail.co.uk> wrote: > >> Paul Rubin wrote: >> >>> Paul E Bennett <Paul_E.Bennett@topmail.co.uk> writes: >>>>> What VMs have you guys used successfully? >>>> For processors like the MSP430 and on up through ARM Cortex families I >>>> would suggest that giving the Forth VM a consideration might be worth >>>> your while. >>> >>> Dave wasn't asking about a VM to run on the target processor, but >>> rather, a way to bundle up a completely configured dev environment >>> involving a lot of third party Windows tools into a blob that the >>> customer could host on their own computer, without having to >>> individually install and configure all the different tools. >>> >>> Forth is an interesting idea in the sense that it replaces all those >>> separate tools with a single, relatively small "do everything" app, >>> avoiding many of the install/configure hassles. But, by the same >>> mechanism, it completely takes over the whole development process >>> including imposing use of a programming language that has some nice >>> properties but is rather far outside the current mainstream. It might >>> be worth looking at but it's not for everyone. >> >> Hadn't considered he would be looking at the other level that is probably >> more suited to Eclipse and the like. Then, being the Forthist I am, I would >> not think to such a gargantuan level as a baseline. Hope he remembers to >> include the CM tools. > > > You're still not thinking big enough. He wants to take an entire > virtual machine image, from the operating system on up, and hand that > disk image file to the customer. Eclipse (or whatever), compilers, > etc., would be installed *in* that virtual machine image. IOW, it's > an image of an entire virtual PC with the entire development > environment installed. The customer then boots that virtual machine > under VMWare, Hyper-V or whatever hypervisor gets picked for this > task, and they've got the exact, and complete, development > environment.
That's just what I'm using. A complete Linux development environment with Eclipse IDE. No licensing hassles and it works well with VMware. The shadow side is that the images are pretty large (about 20 GiB). -- Tauno Voipio
On Sat, 12 Jul 2014 11:20:13 -0700, Dave Nadler wrote:

> So here I am, trying to (again) help (another) customer whose previous > consultant didn't leave complete sources and tools, and who cannot > reproduce their production firmware. Aarrgggg... > > In past, we've required customers to install the tools and do the builds > from our delivered sources, but that is time-consuming for them and > doesn't yield repeatable results for certain toolchain installations > that wander off and fetch mysterious components from the net (and make > it impossible to back up an installation snapshot). > > So, to avoid further repeats, I'm going to do development in a (windows > 7 32-bit) VM with EVERYTHING required. Deliveries will be copies of the > VM and I will ensure the customer actually rebuilds their production > releases in the delivered VM. No more confusion about which version of > tools, libraries, sources, etc.... > > I've avoided doing this in past because of VM problems: > - USB bugginess and/or performance interfering with JTAG dongles, and - > VM performance - annoying 100GB delivery files > > What VMs have you guys used successfully? > The tools for this customer's projects include: > - Microchip MPLABX with ICD-3 debug adapter - TI CCS for MSP430 (GCC > version) using TI MSP-FET430UIF - (phasing out) Rowley Crossworks for > MSP430 using TI adapter > > Host for the VM will be Win7 32-bit, and later Linux and Win7 64-bit. > > I've used QEMU in past for developing inside a Linux VM, and VMware > player for a WinXP machine. I tried VirtualBox but was stymied by USB > issues. But all of these were at least a couple years ago... > > Any advice, suggestions, warnings much appreciated! > Thanks, > Best Regards, Dave
I have found VirtualBox on Ubuntu 12.04 to be almost seamless with what USB stuff I've tried. That's been limited to ST-Link, Olimex Tiny-USB and J-Link, but each one just needed me to select the USB device from the "plug things in" menu and click it. The biggest problems I've had with it are that for all of these I normally use the USB device in question for my own work, and sometimes I'll waste a half an hour debugging a mysterious dongle problem that turns out to be that the VM is running and as a consequence has seized the USB device. I think your idea of delivering the whole VM is a good one. I'm not sure how one can force a customer to use their own builds (I'm very happy with my one customer who insists on doing so, however), but the idea of being able to deliver one closed package and have it work is attractive. I would add that you probably want to verify that builds work correctly when the VM's virtual ethernet port is disconnected, just to make sure that nothing is being fetched from Cyber-Land, although I suspect you've thought of that already. -- Tim Wescott Wescott Design Services http://www.wescottdesign.com
On Sat, 12 Jul 2014 11:20:13 -0700 (PDT), Dave Nadler <drn@nadler.com>
wrote:

>So, to avoid further repeats, I'm going to do development in a >(windows 7 32-bit) VM with EVERYTHING required. Deliveries will >be copies of the VM and I will ensure the customer actually >rebuilds their production releases in the delivered VM. No more >confusion about which version of tools, libraries, sources, >etc. ... > >I've avoided doing this in past because of VM problems: >- USB bugginess and/or performance interfering with JTAG > dongles, and >- VM performance >- annoying 100GB delivery files > >What VMs have you guys used successfully? >The tools for this customer's projects include: >- Microchip MPLABX with ICD-3 debug adapter >- TI CCS for MSP430 (GCC version) using TI MSP-FET430UIF >- (phasing out) Rowley Crossworks for MSP430 using TI adapter > >Host for the VM will be Win7 32-bit, and later Linux and Win7 >64-bit. > >I've used QEMU in past for developing inside a Linux VM, and >VMware player for a WinXP machine. I tried VirtualBox but was >stymied by USB issues. But all of these were at least a couple >years ago... > >Any advice, suggestions, warnings much appreciated! >Thanks, >Best Regards, Dave
I use VmWare - I haven't played much with QEMU, but like you I have had various device issues with VirtualBox. So far VmWare has handled everything I've thrown at it, though if you have to support weird bus cards - or worse conflicting bus cards in different VMs - you may need to use a bare-metal version instead of an OS hosted version. [Coincidentally, VmWare has released the vSphere bare-metal version as freeware: see http://www.vmware.com/products/vsphere-hypervisor/ It's officially unsupported, but help is available through the user community.] You can minimize VM images by not preallocating the virtual disk and by running the defrag/compact tool before archiving the VM image. Further, if multiple toolchains are based on the same OS, you can create a base VM for the OS and separate linked VMs for the unique toolchains [which saves delivering the OS multiple times]. WRT performance: it's a given that you should make sure your virtual disk files are contiguous (the notion that Linux doesn't benefit from defragging is bullshit). Generally a Linux host will outperform a Windows host given the same hardware. If you're on a 64-bit CPU, VmWare will let you run 64-bit clients even on a 32-bit host OS. Hope this helps. George
Tim Wescott <tim@seemywebsite.really> writes:
> [...] > I have found VirtualBox on Ubuntu 12.04 to be almost seamless with what > USB stuff I've tried. That's been limited to ST-Link, Olimex Tiny-USB > and J-Link, but each one just needed me to select the USB device from the > "plug things in" menu and click it.
I second Tim's experience. I've been using VirtualBox under Fedora (10? up to the current 20) for several years and have no complaints (that I can remember...). -- Randy Yates Digital Signal Labs http://www.digitalsignallabs.com
On 13.07.2014 22:52, George Neuner wrote:

> You can minimize VM images by not preallocating the virtual disk and > by running the defrag/compact tool before archiving the VM image.
I was on the delivery side of linux VM images for quite a while. What I did to keep the file-size in check was to run: dd if=/dev/zero of=./nullfile && rm ./nullfile && shudown -h now before delivery. This fills all available free space on your disk-image with zeros. Afterwards the disk/vm image compresses like crazy. (good thing to do anyways - you may not want to leak any deleted data like source-code or emails that found their way into the VM to your customer). Nils
On Tue, 15 Jul 2014 08:33:05 +0200, Nils <n.pipenbrinck@cubic.org>
wrote:

>On 13.07.2014 22:52, George Neuner wrote: > >> You can minimize VM images by not preallocating the virtual disk and >> by running the defrag/compact tool before archiving the VM image. > >I was on the delivery side of linux VM images for quite a while. What I >did to keep the file-size in check was to run: > > dd if=/dev/zero of=./nullfile && rm ./nullfile && shudown -h now > >before delivery. This fills all available free space on your disk-image >with zeros. Afterwards the disk/vm image compresses like crazy.
Yes, you can compress the disk image for delivery, but when you expand a preallocated image for use it will still include those zeroed blocks and be whatever size was configured for the VM. If you don't preallocate, space for the unused blocks doesn't exist and the disk image physically may be much smaller than its logically configured size. This is why I mentioned separating the OS and making the deliverable a linked image - you can quickly create a new linked VM and copy only the data that actually is to be delivered onto its disk.
>(good thing to do anyways - you may not want to leak any deleted data >like source-code or emails that found their way into the VM to your >customer).
A good practice if you intend to clone your actual development disk ... but I wasn't advocating that.
>Nils
George