Greetings, I am trying to get a reasonably fast ADC capability to a PC. I have been told that using the USB bus on a PC has latency, preventing the data from being gathered RT. So I am wondering if an embedded dev board somewhere has fast storage like a SSD that can accumulate data at say, 5-10 mB / s. I will also need a way to continually store the data. The only thing I can think of is to put a SSD on the USB port on something like a Pi, but I don't know if this would be fast enough. The camera which uses the GPU evidently records at 30 fps. Also, I see some activity around the "pru" on BBB. I need to control the ADC with SPI, which could be through the GPIO pins. Now you have to take the above with a grain of salt, since I am SW developer, and just looking for the magic combo of several features: -RT data storage on an SSD or equiv. -SPI control of an ADC -ability to send data to a PC periodically. Anybody ideas appreciated. jb
Does this exist - dev board with GPIO and fast storage to a SSD or equiv.
Started by ●May 22, 2014
Reply by ●May 22, 20142014-05-22
haiticare2011@gmail.com writes:> I am trying to get a reasonably fast ADC capability to a PC. I have > been told that using the USB bus on a PC has latency, preventing the > data from being gathered RT.You mean you have to react to adc events almost instantaneously? USB has an isochronous mode used for stuff like video streaming. It does have some latency, it might be low enough depending on your requirements.> So I am wondering if an embedded dev board somewhere has fast storage > like a SSD that can accumulate data at say, 5-10 mB / s. I will also > need a way to continually store the data. The only thing I can think > of is to put a SSD on the USB port on something like a Pi,The Pi and BBB have SD/microSD card slots as alternative to the USB, though sustained i/o at that speed might be pushing it. There is a new Intel board in the same space, though I think not yet shipping. It has USB3 and SATA: http://www.minnowboard.org/meet-minnowboard-max/
Reply by ●May 22, 20142014-05-22
haiticare2011@gmail.com wrote:> Greetings, > I am trying to get a reasonably fast ADC capability to a PC. I have > been told that using the USB bus on a PC has latency, preventing the > data from being gathered RT. > So I am wondering if an embedded dev board somewhere has fast storage like > a SSD that can accumulate data at say, 5-10 mB / s. I will also need a way > to continually store the data. The only thing I can think of is to put a > SSD on the USB port on something like a Pi, but I don't know if this would > be fast enough. The camera which uses the GPU evidently records at 30 fps. > Also, I see some activity around the "pru" on BBB. > > I need to control the ADC with SPI, which could be through the GPIO pins. > > Now you have to take the above with a grain of salt, since I am SW > developer, and just looking for the magic combo of several features: > -RT data storage on an SSD or equiv. > -SPI control of an ADC > -ability to send data to a PC periodically. > > Anybody ideas appreciated. > > jbHi jb, have a look at this. They have at least done fast the data acquisition. <http://www.element14.com/community/community/knode/single- board_computers/next-gen_beaglebone/blog/2013/08/04/bbb--high-speed-data- acquisition-and-web-based-ui> Perhaps you could use the ADC of the BBB itself. Here you find some specs of it. <http://beaglebone.cameon.net/home/reading-the-analog-inputs-adc> I am not sure if the SD-card interface and the card itself can support the sustained writing speed you want. But you use a PC connected thru Ethernet for this. Since the BBB has a 100MBit/s Ethernet it should be possible to just push the data over to some storage or PC. Cheers Reinhardt
Reply by ●May 22, 20142014-05-22
On Thu, 22 May 2014 14:47:31 -0700, haiticare2011 wrote:> Greetings, > I am trying to get a reasonably fast ADC capability to a PC. I have been > told that using the USB bus on a PC has latency, preventing the data > from being gathered RT. > So I am wondering if an embedded dev board somewhere has fast storage > like a SSD that can accumulate data at say, 5-10 mB / s. I will also > need a way to continually store the data. The only thing I can think of > is to put a SSD on the USB port on something like a Pi, but I don't know > if this would be fast enough. > The camera which uses the GPU evidently records at 30 fps. Also, I see > some activity around the "pru" on BBB. > > I need to control the ADC with SPI, which could be through the GPIO > pins. > > Now you have to take the above with a grain of salt, since I am SW > developer, and just looking for the magic combo of several features: > -RT data storage on an SSD or equiv. > -SPI control of an ADC -ability to send data to a PC periodically. > > Anybody ideas appreciated.If you're storing it off to disk you're not using it real time. What's your real concern? A decent data acquisition system, with or without USB, should be able to capture everything coming in. -- Tim Wescott Wescott Design Services http://www.wescottdesign.com
Reply by ●May 23, 20142014-05-23
haiticare2011@gmail.com wrote:> So I am wondering if an embedded dev board somewhere has fast storage like > a SSD that can accumulate data at say, 5-10 mB / s. I will also need a > way to continually store the data. The only thing I can think of is to > put a SSD on the USB port on something like a Pi, but I don't know if this > would be fast enough. The camera which uses the GPU evidently records at > 30 fps. Also, I see some activity around the "pru" on BBB.I'd be tempted to use a SATA SSD. They're bigger, cheaper (per GB), more robust, and less prone to performance variability if you buy the wrong one (you don't go from 1000 IOPS to 10 IOPS like you do if you buy the wrong SD card). If you buy a USB <thing> it's not doing USB natively, it'll have a bridge chip the other side of the USB interface, which adds more latency. While SATA makes no guarantees about latency, it's much less work from a CPU perspective than USB. I haven't been keeping up with all the new boards, but some of the ARM boards that run Linux have SATA ports and GPIO. Theo
Reply by ●May 23, 20142014-05-23
Theo Markettos <theom+news@chiark.greenend.org.uk> writes:> I haven't been keeping up with all the new boards, but some of the ARM > boards that run Linux have SATA ports and GPIO.I have no personal experience in this area but a hardware guy warned me recently that complex ARM boards tend to be full of software bugs in the Linux drivers due to corner cases in the specific hardware, etc. He said the Raspberry Pi is pretty reliable because so many were in use that the bugs have mostly been found and fixed by now, and the Beaglebone Black was #2 in number of units deployed, but for the others you should expect to spend some of your development time on Linux kernel debugging. I don't know if this applies to the Minnowboard that I mentioned, which is an x86 board rather than an ARM, and I think it is designed by Intel rather than by hobbyists.
Reply by ●May 23, 20142014-05-23
On 22/05/14 23:47, haiticare2011@gmail.com wrote:> Greetings, > I am trying to get a reasonably fast ADC capability to a PC. I have > been told that using the USB bus on a PC has latency, preventing the > data from being gathered RT.No, you have not been told that. In all your posts in sci.electronics.design, you have been told that USB can have significant latency depending on how it is used - but you have also been told how it can work fine for the job you describe. USB latency can be an issue if you need fast turnaround (read something, then write something new depending on what you read) - for continuous reads of an ADC, there is no problem. You have also been warned not to connect other devices to the same USB bus. You have also regularly banded around the term "RT", without showing any understanding of what "real time" actually means. It does not mean "as fast as possible", or "as fast as I can imagine", or "as fast as randomly picked numbers that may or may not apply to some parts of the system". It /really/ means "within a guaranteed pre-determined time limit". Getting back to reality, you have been thinking of using one or more ADS8344 ADCs. These have a maximum rate of 100 KSps, at 16-bit. A 2Mbps (Mb is "Mega bit" - mB is "milli byte") SPI bus will saturate the device (even allowing for some overhead). But even if you have 4 of these devices on a common SPI bus, an FTDI USB-to-SPI module will handle them all. The USB latency will mean a slight delay between starting and receiving the first input, but if you do your PC-side software correctly then you should be able to get continuous sampling, with data coming in at 800 KB/s.> So I am wondering if an embedded dev board somewhere has fast storage like a SSD > that can accumulate data at say, 5-10 mB / s.Why do you think you need an SSD to get 10 MB/s? (I assume you mean 10 MB/s, i.e., 10 Megabyte per second, rather than 10 millibytes per second.) Even a decent USB memory stick will cope with that - it is certainly not a challenge for a hard disk, an good network, embedded NAND, etc. And of course it is 10 times the rate of the ADC (guessing 4 chips in parallel). And this is all pointless unless you can say how much data you need in total. You really need to start figuring out what data you actually want to collect, and what sampling rates you have there. Once you have realistic values there, you can start to work out what you need in order to collect and process that data. Randomly picking parts of the chain with throughput rates differing by several orders of magnitude will not help.> I will also need a way to > continually store the data. The only thing I can think of is to put a SSD on > the USB port on something like a Pi, but I don't know if this would be fast enough.You don't know if it will be fast enough, because you don't know what "fast enough" is. An SSD on USB 2.0 will peak a bit lower than 60 MBps, since that is the limit for USB 2.0. It is the same rate as you will get with a hard disk on USB 2.0.> The camera which uses the GPU evidently records at 30 fps. Also, I see some activity around the "pru" on BBB.What camera? What GPU? And so what?> > I need to control the ADC with SPI, which could be through the GPIO pins.You think you need SSD speeds to store the data, but GPIO bit-banging is good enough for your SPI bus?> > Now you have to take the above with a grain of salt, since I am SW developer, and > just looking for the magic combo of several features: > -RT data storage on an SSD or equiv. > -SPI control of an ADC > -ability to send data to a PC periodically. > > Anybody ideas appreciated. >Here's the best idea - find yourself a systems designer who can specify the system and get a realistic picture of how the parts will fit together. You've asked repeatedly about this in s.e.d., and leapt between a USB module, PCI express cards, microcontroller boards, and high-spec PCs, with corresponding leaps in the data throughput you think you need - without ever giving useful concrete information about your requirements. And now you have jumped back to the beginning again in c.a.e. So find someone who knows about hardware, and knows about the task you want. Talk it over with them, draw pictures on a whiteboard, and get the requirements clear. Once you have that in place, come here and ask again - then you can get useful help.> jb >
Reply by ●May 23, 20142014-05-23
On Friday, May 23, 2014 4:30:11 AM UTC-4, David Brown wrote:> On 22/05/14 23:47, zhaiticare2011@gmail.com wrote: > > > Greetings, > > > I am trying to get a reasonably fast ADC capability to a PC. I have > > > been told that using the USB bus on a PC has latency, preventing the > > > data from being gathered RT. > > > > No, you have not been told that. In all your posts in > > sci.electronics.design, you have been told that USB can have significant > > latency depending on how it is used - but you have also been told how it > > can work fine for the job you describe. USB latency can be an issue if > > you need fast turnaround (read something, then write something new > > depending on what you read) - for continuous reads of an ADC, there is > > no problem. You have also been warned not to connect other devices to > > the same USB bus. > > > > You have also regularly banded around the term "RT", without showing any > > understanding of what "real time" actually means. It does not mean "as > > fast as possible", or "as fast as I can imagine", or "as fast as > > randomly picked numbers that may or may not apply to some parts of the > > system". It /really/ means "within a guaranteed pre-determined time limit". > > > > Getting back to reality, you have been thinking of using one or more > > ADS8344 ADCs. These have a maximum rate of 100 KSps, at 16-bit. A > > 2Mbps (Mb is "Mega bit" - mB is "milli byte") SPI bus will saturate the > > device (even allowing for some overhead). But even if you have 4 of > > these devices on a common SPI bus, an FTDI USB-to-SPI module will handle > > them all. The USB latency will mean a slight delay between starting and > > receiving the first input, but if you do your PC-side software correctly > > then you should be able to get continuous sampling, with data coming in > > at 800 KB/s. > > > > > So I am wondering if an embedded dev board somewhere has fast storage like a SSD > > > that can accumulate data at say, 5-10 mB / s. > > > > Why do you think you need an SSD to get 10 MB/s? (I assume you mean 10 > > MB/s, i.e., 10 Megabyte per second, rather than 10 millibytes per > > second.) Even a decent USB memory stick will cope with that - it is > > certainly not a challenge for a hard disk, an good network, embedded > > NAND, etc. And of course it is 10 times the rate of the ADC (guessing 4 > > chips in parallel). > > > > And this is all pointless unless you can say how much data you need in > > total. > > > > > > You really need to start figuring out what data you actually want to > > collect, and what sampling rates you have there. Once you have > > realistic values there, you can start to work out what you need in order > > to collect and process that data. Randomly picking parts of the chain > > with throughput rates differing by several orders of magnitude will not > > help. > > > > > > > I will also need a way to > > > continually store the data. The only thing I can think of is to put a SSD on > > > the USB port on something like a Pi, but I don't know if this would be fast enough. > > > > You don't know if it will be fast enough, because you don't know what > > "fast enough" is. An SSD on USB 2.0 will peak a bit lower than 60 MBps, > > since that is the limit for USB 2.0. It is the same rate as you will > > get with a hard disk on USB 2.0. > > > > > The camera which uses the GPU evidently records at 30 fps. Also, I see some activity around the "pru" on BBB. > > > > What camera? What GPU? And so what? > > > > > > > > I need to control the ADC with SPI, which could be through the GPIO pins. > > > > You think you need SSD speeds to store the data, but GPIO bit-banging is > > good enough for your SPI bus? > > > > > > > > Now you have to take the above with a grain of salt, since I am SW developer, and > > > just looking for the magic combo of several features: > > > -RT data storage on an SSD or equiv. > > > -SPI control of an ADC > > > -ability to send data to a PC periodically. > > > > > > Anybody ideas appreciated. > > > > > > > Here's the best idea - find yourself a systems designer who can specify > > the system and get a realistic picture of how the parts will fit > > together. You've asked repeatedly about this in s.e.d., and leapt > > between a USB module, PCI express cards, microcontroller boards, and > > high-spec PCs, with corresponding leaps in the data throughput you think > > you need - without ever giving useful concrete information about your > > requirements. And now you have jumped back to the beginning again in c.a.e. > > > > So find someone who knows about hardware, and knows about the task you > > want. Talk it over with them, draw pictures on a whiteboard, and get > > the requirements clear. Once you have that in place, come here and ask > > again - then you can get useful help. > > > > > > > > > jb > > >David, Actually, I am getting useful help. And you have been a purveyor of some of that. As I was at pains to point out, and did, the speed of acquisition is open-ended - the spec is "as much as is practical for my situation." IOW, I'd like to do this for less than 1k$ right now. Alhough your manner is one of constant exasperation, you have given me useful leads. :) I have downloaded all the datasheets for the FT4232 dev board - and the MSSE docs. But you must realize, opinion onions have various layers, and I have received much good info here, though most of it is partial and not due to my lack of precsion, IMO. The situation is like SETI - Those guys don't know what BW the signal has they are searching for. In my case, we are going to be sensing medical diagnostic situations, and I can't a priori specify what the data through-put will be, except that the process will reveal it. (See the quote below.) That you stick to your guns about USB latency is a good sign for your FTDI solution, BUT I'm still concerned about the OS handling the data and storing it quickly. John "If you are not confused at the start of a project, you are doing something wrong." - Jim Williams, quoting R Feynman.
Reply by ●May 23, 20142014-05-23
On 23/05/14 14:32, haiticare2011@gmail.com wrote:> On Friday, May 23, 2014 4:30:11 AM UTC-4, David Brown wrote: >> On 22/05/14 23:47, zhaiticare2011@gmail.com wrote: >> >>> Greetings, >> >>> I am trying to get a reasonably fast ADC capability to a PC. I have >> >>> been told that using the USB bus on a PC has latency, preventing the >> >>> data from being gathered RT. >> >> >> >> No, you have not been told that. In all your posts in >> >> sci.electronics.design, you have been told that USB can have significant >> >> latency depending on how it is used - but you have also been told how it >> >> can work fine for the job you describe. USB latency can be an issue if >> >> you need fast turnaround (read something, then write something new >> >> depending on what you read) - for continuous reads of an ADC, there is >> >> no problem. You have also been warned not to connect other devices to >> >> the same USB bus. >> >> >> >> You have also regularly banded around the term "RT", without showing any >> >> understanding of what "real time" actually means. It does not mean "as >> >> fast as possible", or "as fast as I can imagine", or "as fast as >> >> randomly picked numbers that may or may not apply to some parts of the >> >> system". It /really/ means "within a guaranteed pre-determined time limit". >> >> >> >> Getting back to reality, you have been thinking of using one or more >> >> ADS8344 ADCs. These have a maximum rate of 100 KSps, at 16-bit. A >> >> 2Mbps (Mb is "Mega bit" - mB is "milli byte") SPI bus will saturate the >> >> device (even allowing for some overhead). But even if you have 4 of >> >> these devices on a common SPI bus, an FTDI USB-to-SPI module will handle >> >> them all. The USB latency will mean a slight delay between starting and >> >> receiving the first input, but if you do your PC-side software correctly >> >> then you should be able to get continuous sampling, with data coming in >> >> at 800 KB/s. >> >> >> >>> So I am wondering if an embedded dev board somewhere has fast storage like a SSD >> >>> that can accumulate data at say, 5-10 mB / s. >> >> >> >> Why do you think you need an SSD to get 10 MB/s? (I assume you mean 10 >> >> MB/s, i.e., 10 Megabyte per second, rather than 10 millibytes per >> >> second.) Even a decent USB memory stick will cope with that - it is >> >> certainly not a challenge for a hard disk, an good network, embedded >> >> NAND, etc. And of course it is 10 times the rate of the ADC (guessing 4 >> >> chips in parallel). >> >> >> >> And this is all pointless unless you can say how much data you need in >> >> total. >> >> >> >> >> >> You really need to start figuring out what data you actually want to >> >> collect, and what sampling rates you have there. Once you have >> >> realistic values there, you can start to work out what you need in order >> >> to collect and process that data. Randomly picking parts of the chain >> >> with throughput rates differing by several orders of magnitude will not >> >> help. >> >> >> >> >> >>> I will also need a way to >> >>> continually store the data. The only thing I can think of is to put a SSD on >> >>> the USB port on something like a Pi, but I don't know if this would be fast enough. >> >> >> >> You don't know if it will be fast enough, because you don't know what >> >> "fast enough" is. An SSD on USB 2.0 will peak a bit lower than 60 MBps, >> >> since that is the limit for USB 2.0. It is the same rate as you will >> >> get with a hard disk on USB 2.0. >> >> >> >>> The camera which uses the GPU evidently records at 30 fps. Also, I see some activity around the "pru" on BBB. >> >> >> >> What camera? What GPU? And so what? >> >> >> >>> >> >>> I need to control the ADC with SPI, which could be through the GPIO pins. >> >> >> >> You think you need SSD speeds to store the data, but GPIO bit-banging is >> >> good enough for your SPI bus? >> >> >> >>> >> >>> Now you have to take the above with a grain of salt, since I am SW developer, and >> >>> just looking for the magic combo of several features: >> >>> -RT data storage on an SSD or equiv. >> >>> -SPI control of an ADC >> >>> -ability to send data to a PC periodically. >> >>> >> >>> Anybody ideas appreciated. >> >>> >> >> >> >> Here's the best idea - find yourself a systems designer who can specify >> >> the system and get a realistic picture of how the parts will fit >> >> together. You've asked repeatedly about this in s.e.d., and leapt >> >> between a USB module, PCI express cards, microcontroller boards, and >> >> high-spec PCs, with corresponding leaps in the data throughput you think >> >> you need - without ever giving useful concrete information about your >> >> requirements. And now you have jumped back to the beginning again in c.a.e. >> >> >> >> So find someone who knows about hardware, and knows about the task you >> >> want. Talk it over with them, draw pictures on a whiteboard, and get >> >> the requirements clear. Once you have that in place, come here and ask >> >> again - then you can get useful help. >> >> >> >> >> >> >> >>> jb >> >>> > > David, > Actually, I am getting useful help. And you have been a purveyor of some of > that. As I was at pains to point out, and did, the speed of acquisition is > open-ended - the spec is "as much as is practical for my situation." IOW, I'd > like to do this for less than 1k$ right now. > > Alhough your manner is one of constant exasperation, you have given me useful > leads. :) I have downloaded all the datasheets for the FT4232 dev board - and > the MSSE docs. > > But you must realize, opinion onions have various layers, and I have received > much good info here, though most of it is partial and not due to my lack of > precsion, IMO. > > The situation is like SETI - Those guys don't know what BW the signal has they > are searching for. In my case, we are going to be sensing medical diagnostic > situations, and I can't a priori specify what the data through-put will be, > except that the process will reveal it. (See the quote below.) > > That you stick to your guns about USB latency is a good sign for your FTDI > solution, BUT I'm still concerned about the OS handling the data and storing it > quickly. > > John >Pick a speed you like, buy a pre-made data acquisition system, and try it out. Once you have sample data there, you can decide whether that speed was fast enough, or too slow. /Then/ you can start specifying what you want in a /real/ system, and start putting together something that will be more cost-effective in quantity and better tuned to the final usage. At the moment, you are guessing a ADCs with a total data rate of 1 MBps, an SSD that can handle 300 MBps connected to a USB interface at about 50 MBps - you are all over the place. There is a reason this stuff is called "Research and Development". Do the research, then you can start of the development. And don't guess about OS data handling being "a concern" - either listen to people who have told you what they know, or try experiments yourself. And note that /you/ are the person who has been insisting on using a PC because it has high throughput, high processing speed, and high quality OS's and development tools. A $2 microcontroller can handle the speeds you have at your current bottleneck - a PC should not have a problem storing this data. (Yes, I am in constant exasperation here. I am trying to help, but this is /not/ how you should be developing any sort of project.)
Reply by ●May 23, 20142014-05-23
Paul Rubin <no.email@nospam.invalid> wrote:> I have no personal experience in this area but a hardware guy warned me > recently that complex ARM boards tend to be full of software bugs in the > Linux drivers due to corner cases in the specific hardware, etc. He > said the Raspberry Pi is pretty reliable because so many were in use > that the bugs have mostly been found and fixed by now, and the > Beaglebone Black was #2 in number of units deployed, but for the others > you should expect to spend some of your development time on Linux kernel > debugging.That is a risk. I'm especially wary of the chips with non-complete public datasheets (eg NVIDIA, Allwinner) because you can't just dive in and fix things. The kernel engineering of some of the SoCs is patchy too (vendor releases sources for 2.6.29 or something, it's a big effort to port to modern 3.1x, maybe some of the drivers like graphics or wifi are binary-only).> I don't know if this applies to the Minnowboard that I > mentioned, which is an x86 board rather than an ARM, and I think it is > designed by Intel rather than by hobbyists.I have a Galileo board, which is the previous Intel idea to the Minnowboard. It's utterly utterly terrible for IO. There are 2 (yes two) high speed GPIO lines - the rest are all via I2C expanders, max speed 240Hz. If that's not your cup of tea you have USB or PCIe - which are fine, but then they dress it up in an Arduino formfactor and pretend it's a microcontroller. Plus they don't run standard i386 Linux distros, so you can't just boot Debian on the thing like you can with most of the ARM boards. So the primary reason for picking Intel (binary compatibility and the large x86 ecosystem) is gone. IMO Intel have completely lost the plot as far as embedded goes - if it isn't PC architecture they don't know how to handle it. They're currently making plans for IoT but they haven't a hope. IMHO anyway. Theo







