On Mon, 05 Dec 2005 12:52:27 +1100, Mark McDougall wrote:> Keith wrote: > >> Memory on the PCI bus can no longer cached (deprecated in V2.2, >> IIRC). Cacheable memory on the PCI bus is a mess so most systems, >> even before it was yanked out of the spec, didn't support it. > > I'm not sure where you got this info, but it's news to me! :OThe specs, perhaps?> To answer the OP: > > As for mapping PCI memory from windows, there's a toolkit called TVICPCI > <http://entechtaiwan.net/dev/pci/index.shtm> which grants user space > access to PCI resources, which of course includes memory. They have a > demo version for evaluation. You won't be able to tell Windows to use > this as generic memory space (ie. Windows won't be executing out of it), > but you will be able to use it for your own data. > > There's a bit in the PCI BAR that specifies whether or not the region is > pre-fetchable. If set, a PCI master will know that it is allowed to use > MRL (memory read line) or MRM (memory read multiple) on that address > space. That doesn't necessarily mean it will.That doesn't change the cacheability. Caches are a processor thing and *NOT* under control of any PCI device. -- Keith
pci and caching
Started by ●December 3, 2005
Reply by ●December 4, 20052005-12-04
Reply by ●December 5, 20052005-12-05
Keith wrote:> That doesn't change the cacheability. Caches are a processor thing > and *NOT* under control of any PCI device.Which begs the question, why would the PCI spec refer to something that has nothing to do with PCI? If a PCI memory space is marked as 'pre-fetchable' then it guarantees, among other things, that the act of pre-fetching memory has no side-effects. This means nothing more than the fact that it may be a suitable candidate for caching, if the platform supports it. In this case, a master may issue MRL (& MRM) commands. OTOH, cache-coherency (which I assume you're hinting at) is a different problem altogether - especially if you've got multiple bus masters accessing PCI memory space with their own caches. However, this is a *system* problem and (IMHO) not really any concern of the PCI bus spec group to mandate that PCI memory is not 'cacheable' - whatever that means in each context! In fact, there's little discussion what-so-ever in the spec (that I can see) about 'caches' - which is just what I would expect. BTW I'm quite happy to be shown the error in my reasoning! Regards, Mark
Reply by ●December 5, 20052005-12-05
On Mon, 05 Dec 2005 15:01:25 +1100, Mark McDougall <markm@vl.com.au> wrote:>Keith wrote: > >> That doesn't change the cacheability. Caches are a processor thing >> and *NOT* under control of any PCI device. > >Which begs the question, why would the PCI spec refer to something that >has nothing to do with PCI? > >If a PCI memory space is marked as 'pre-fetchable' then it guarantees, >among other things, that the act of pre-fetching memory has no >side-effects. This means nothing more than the fact that it may be a >suitable candidate for caching, if the platform supports it. In this >case, a master may issue MRL (& MRM) commands. > >OTOH, cache-coherency (which I assume you're hinting at) is a different >problem altogether - especially if you've got multiple bus masters >accessing PCI memory space with their own caches. However, this is a >*system* problem and (IMHO) not really any concern of the PCI bus spec >group to mandate that PCI memory is not 'cacheable' - whatever that >means in each context! > >In fact, there's little discussion what-so-ever in the spec (that I can >see) about 'caches' - which is just what I would expect. >But only the PCI card itself knows whether its memory-space registers are suitable for caching. If it's video memory, likely they are; if's an ADC buffer, it sure ain't. Seems to me that, if PCI space is allowed to be cached, there should be a mechanism that allows a PCI card to tell the bios (or OS) whether caching is a good idea for it. John
Reply by ●December 5, 20052005-12-05
Mark McDougall wrote:> Keith wrote: > > > That doesn't change the cacheability. Caches are a processor thing > > and *NOT* under control of any PCI device. > > Which begs the question, why would the PCI spec refer to something that > has nothing to do with PCI? > > If a PCI memory space is marked as 'pre-fetchable' then it guarantees, > among other things, that the act of pre-fetching memory has no > side-effects. This means nothing more than the fact that it may be a > suitable candidate for caching, if the platform supports it. In this > case, a master may issue MRL (& MRM) commands. > > OTOH, cache-coherency (which I assume you're hinting at) is a different > problem altogether - especially if you've got multiple bus masters > accessing PCI memory space with their own caches. However, this is a > *system* problem and (IMHO) not really any concern of the PCI bus spec > group to mandate that PCI memory is not 'cacheable' - whatever that > means in each context! > > In fact, there's little discussion what-so-ever in the spec (that I can > see) about 'caches' - which is just what I would expect. > > BTW I'm quite happy to be shown the error in my reasoning! >Don't confuse pre-fetching with caching. And don't confuse MRM with cachable. To be able to pre-fetch data simply requires that a read to that data not cause any side-effect. To be able to cache data requires that not outputting that data not cause any side-effect. Generally the only cachable device is generic RAM. When you output to any other device besides RAM you expect to see that output in the real world. RAM is therefore the only safe device to cache. For example, say you have a memory mapped variable 'foo'. You need to send a signal to the device that 'foo' belongs to and it needs to be a pulse. Say you code something like: foo = 1; delay(1); foo = 0; If 'foo' was cached, then the signal may or may not be generated in this case. Even with the delay statement (which prevents some C compilers form optimising away foo=1) 'foo' may not be written back to the device if there is still enough space in the cache to not require writing back 'foo'. Pre-fetchable means it is safe to read data multiple times. Cachable means it is safe to not write out data.
Reply by ●December 5, 20052005-12-05
I guess the answer for linux is hidden between ; "Chapter 9 page 236 I/O Registers and Conventional Memory" and "Chapter 12 page 316 Accessing the I/O and Memory Spaces" of ldd3. yusuf
Reply by ●December 5, 20052005-12-05
In article <4393bb75$0$23327$5a62ac22@per-qv1-newsreader- 01.iinet.net.au>, markm@vl.com.au says...> Keith wrote: > > > That doesn't change the cacheability. Caches are a processor thing > > and *NOT* under control of any PCI device. > > Which begs the question, why would the PCI spec refer to something that > has nothing to do with PCI?Short answer: Snoops from any PCI initiator into PCI cached memory are in the purview of the spec. ;-) Longer answer[*]: The SBO# (Snoop Back Off) and SDONE (snoop done) signals are part of the pre-PCI2.2 spec (actually, I believe in 2.2 it's recommended that they not be used and pulled high). These are used by the memory bridge to initiate retries to cached memory. SDONE indicates an access to cached memory is complete. SBO# active indicates a cached line is being accessed and the access must be terminated by the initiator and retried later. [*] I've ever used these things so I'm not real up on the spec here. The performance is horrible so isn't often implemented and less often used.> If a PCI memory space is marked as 'pre-fetchable' then it guarantees, > among other things, that the act of pre-fetching memory has no > side-effects. This means nothing more than the fact that it may be a > suitable candidate for caching, if the platform supports it. In this > case, a master may issue MRL (& MRM) commands.prefetching <> cacheing> OTOH, cache-coherency (which I assume you're hinting at) is a different > problem altogether - especially if you've got multiple bus masters > accessing PCI memory space with their own caches. However, this is a > *system* problem and (IMHO) not really any concern of the PCI bus spec > group to mandate that PCI memory is not 'cacheable' - whatever that > means in each context!But it *is* part of the (pre 2.2) spec. The bus must guarantee coherency in this case. The way it does it is with back-offs and retries. Now think about this with multiple bridges and initiators. It gets to be a mess.> In fact, there's little discussion what-so-ever in the spec (that I can > see) about 'caches' - which is just what I would expect.It's there in the older versions of the spec. As I've mentioned, it's been deprecated in later versions.> BTW I'm quite happy to be shown the error in my reasoning!-- Keith
Reply by ●December 5, 20052005-12-05
On Sat, 03 Dec 2005 09:31:04 -0800, John Larkin <jjlarkin@highNOTlandTHIStechnologyPART.com> wrote:>Consider a Pentium PC with main memory and a PCI bus. One can plug >memory-type devices into the PCI bus, things like video or ADC >buffers, CPCI cards, and including, I suppose, more program-space >memory. > >A couple of questions: > >Is there (I guess there must be) a mechanism for a Windows program to >directly map a chunk of PCI-bus memory into its virtual address space? >Anybody know how this works?With Windows NT (2k/xp/2k3) applications can never access hardware directly. (*) You will have to write a kernel mode device driver to control your piece of hardware. The NT kernel mode api is very different from what you know from the user mode Windows api. You will need the Microsoft Platform SDK and the DDK and the Microsoft C compiler. The DDK contains samples and documentation for everything. Try to minimize the user/kernel mode switches. (*) There are kludges like giveio.sys etc, which allow user applications to access io ports. Forget about these - it won't be enough for you. Mit freundlichen Gr��en Frank-Christian Kr�gel
Reply by ●December 5, 20052005-12-05
John Larkin wrote:> Does anybody know how the BIOS decides what should be cached? There's > nothing in a device's PCI config registers that says "don't cache me" > as far as I can tell.The BIOS doesn't. The device driver (which knows the PCI card intimately) asks for non-cached memory when it asks for a virtual mapping of the physical address range of the card. If that's appropriate. If there's no reason why it can't be cached, it doesn't. Steve -------------------------------------------------------------------------------------------- Steve Schefter phone: +1 705 725 9999 x26 The Software Group Limited fax: +1 705 725 9666 642 Welham Road, Barrie, Ontario CANADA L4N 9A1 Web: www.wanware.com
Reply by ●December 5, 20052005-12-05
On 5 Dec 2005 13:31:00 -0800, steve_schefter@hotmail.com wrote:>John Larkin wrote: >> Does anybody know how the BIOS decides what should be cached? There's >> nothing in a device's PCI config registers that says "don't cache me" >> as far as I can tell. > >The BIOS doesn't. The device driver (which knows the PCI card >intimately) asks for non-cached memory when it asks for a virtual >mapping of the physical address range of the card. If that's >appropriate. If there's no reason why it can't be cached, it doesn't. > > Steve > >-------------------------------------------------------------------------------------------- >Steve Schefter phone: +1 705 725 9999 x26 >The Software Group Limited fax: +1 705 725 9666 >642 Welham Road, >Barrie, Ontario CANADA L4N 9A1 Web: www.wanware.comBut we often use PCI devices under DOS, with no device driver at all. It appears to me that the BIOS locates devices in PCI config space, looks at the requested resources (in the PCI overhead reggies) and assigns memory space, as requested, to the gadgets. Usually these addresses are really high, past 2G as I recall. So the cached/uncached situation must be resolved, somehow, before the os boots, although it can certainly be changed by drivers later. John
Reply by ●December 5, 20052005-12-05
yusufilker@gmail.com wrote:> I guess the answer for linux is hidden between ; > > > "Chapter 9 page 236 I/O Registers and Conventional Memory" > > > and > > > "Chapter 12 page 316 Accessing the I/O and Memory Spaces" >What book? The usual answer for Linux is O'Reilly's Linux Device Driver, chapters 7, 8 and 13. The free online version can be found at: http://www.xml.com/ldd/chapter/book/