47

Is there any API for determining the physical address from virtual address in Linux operating system?

3

5 Answers 5

40

Kernel and user space work with virtual addresses (also called linear addresses) that are mapped to physical addresses by the memory management hardware. This mapping is defined by page tables, set up by the operating system.

DMA devices use bus addresses. On an i386 PC, bus addresses are the same as physical addresses, but other architectures may have special address mapping hardware to convert bus addresses to physical addresses.

In Linux, you can use these functions from asm/io.h:

  • virt_to_phys(virt_addr);
  • phys_to_virt(phys_addr);
  • virt_to_bus(virt_addr);
  • bus_to_virt(bus_addr);

All this is about accessing ordinary memory. There is also "shared memory" on the PCI or ISA bus. It can be mapped inside a 32-bit address space using ioremap(), and then used via the readb(), writeb() (etc.) functions.

Life is complicated by the fact that there are various caches around, so that different ways to access the same physical address need not give the same result.

Also, the real physical address behind virtual address can change. Even more than that - there could be no address associated with a virtual address until you access that memory.

As for the user-land API, there are none that I am aware of.

6
  • Thanks for your reply. I wonder why there is no user-land API. What is the complexity in providing that API in Linux ? Is there any script or tool that provides the physical address corresponding to the virtual address ? Apr 22, 2011 at 3:05
  • 2
    @S.Man: It doesn't matter whether there is a user-land API for this or not. The thing that is more important is - what can that user-land API do for you? In my opinion, it is useless. And so many other people think I believe. Why would one invest their time and efforts to do useless things?
    – user405725
    Apr 23, 2011 at 3:06
  • 3
    As for the user-land API, 1 way I can think about is to code the kernel space functions, and let the user space to query (get values). The motivation to get the physical address may be for some malware detection usage. Note to others: Do not think it is useless, just because you cannot think of a use for it.
    – Ursa Major
    Feb 11, 2014 at 0:13
  • Note that virt_to_phys only works for kmalloc memory: stackoverflow.com/a/45128487/895245 Aug 4, 2017 at 5:16
  • 1
    I lloked at the virt_to_phy function and all it does is to subtract IDNET_ADDRESS instead of doing a pagetable walk, Im assuming that not a bug (otherwise would have probably been discovered by now) can you explain this?
    – tomer.z
    Mar 15, 2018 at 15:36
38

/proc/<pid>/pagemap userland minimal runnable example

virt_to_phys_user.c

#define _XOPEN_SOURCE 700
#include <fcntl.h> /* open */
#include <stdint.h> /* uint64_t  */
#include <stdio.h> /* printf */
#include <stdlib.h> /* size_t */
#include <unistd.h> /* pread, sysconf */

typedef struct {
    uint64_t pfn : 55;
    unsigned int soft_dirty : 1;
    unsigned int file_page : 1;
    unsigned int swapped : 1;
    unsigned int present : 1;
} PagemapEntry;

/* Parse the pagemap entry for the given virtual address.
 *
 * @param[out] entry      the parsed entry
 * @param[in]  pagemap_fd file descriptor to an open /proc/pid/pagemap file
 * @param[in]  vaddr      virtual address to get entry for
 * @return 0 for success, 1 for failure
 */
int pagemap_get_entry(PagemapEntry *entry, int pagemap_fd, uintptr_t vaddr)
{
    size_t nread;
    ssize_t ret;
    uint64_t data;
    uintptr_t vpn;

    vpn = vaddr / sysconf(_SC_PAGE_SIZE);
    nread = 0;
    while (nread < sizeof(data)) {
        ret = pread(pagemap_fd, ((uint8_t*)&data) + nread, sizeof(data) - nread,
                vpn * sizeof(data) + nread);
        nread += ret;
        if (ret <= 0) {
            return 1;
        }
    }
    entry->pfn = data & (((uint64_t)1 << 55) - 1);
    entry->soft_dirty = (data >> 55) & 1;
    entry->file_page = (data >> 61) & 1;
    entry->swapped = (data >> 62) & 1;
    entry->present = (data >> 63) & 1;
    return 0;
}

/* Convert the given virtual address to physical using /proc/PID/pagemap.
 *
 * @param[out] paddr physical address
 * @param[in]  pid   process to convert for
 * @param[in] vaddr virtual address to get entry for
 * @return 0 for success, 1 for failure
 */
int virt_to_phys_user(uintptr_t *paddr, pid_t pid, uintptr_t vaddr)
{
    char pagemap_file[BUFSIZ];
    int pagemap_fd;

    snprintf(pagemap_file, sizeof(pagemap_file), "/proc/%ju/pagemap", (uintmax_t)pid);
    pagemap_fd = open(pagemap_file, O_RDONLY);
    if (pagemap_fd < 0) {
        return 1;
    }
    PagemapEntry entry;
    if (pagemap_get_entry(&entry, pagemap_fd, vaddr)) {
        return 1;
    }
    close(pagemap_fd);
    *paddr = (entry.pfn * sysconf(_SC_PAGE_SIZE)) + (vaddr % sysconf(_SC_PAGE_SIZE));
    return 0;
}

int main(int argc, char **argv)
{
    pid_t pid;
    uintptr_t vaddr, paddr = 0;

    if (argc < 3) {
        printf("Usage: %s pid vaddr\n", argv[0]);
        return EXIT_FAILURE;
    }
    pid = strtoull(argv[1], NULL, 0);
    vaddr = strtoull(argv[2], NULL, 0);
    if (virt_to_phys_user(&paddr, pid, vaddr)) {
        fprintf(stderr, "error: virt_to_phys_user\n");
        return EXIT_FAILURE;
    };
    printf("0x%jx\n", (uintmax_t)paddr);
    return EXIT_SUCCESS;
}

GitHub upstream.

Usage:

sudo ./virt_to_phys_user.out <pid> <virtual-address>

sudo is required to read /proc/<pid>/pagemap even if you have file permissions as explained at: https://unix.stackexchange.com/questions/345915/how-to-change-permission-of-proc-self-pagemap-file/383838#383838

As mentioned at: https://stackoverflow.com/a/46247716/895245 Linux allocates page tables lazily, so make sure that you read and write a byte to that address from the test program before using virt_to_phys_user.

How to test it out

Test program:

#define _XOPEN_SOURCE 700
#include <stdint.h>
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>

enum { I0 = 0x12345678 };

static volatile uint32_t i = I0;

int main(void) {
    printf("vaddr %p\n", (void *)&i);
    printf("pid %ju\n", (uintmax_t)getpid());
    while (i == I0) {
        sleep(1);
    }
    printf("i %jx\n", (uintmax_t)i);
    return EXIT_SUCCESS;
}

The test program outputs the address of a variable it owns, and its PID, e.g.:

vaddr 0x600800
pid 110

and then you can pass convert the virtual address with:

sudo ./virt_to_phys_user.out 110 0x600800

Finally, the conversion can be tested by using /dev/mem to observe / modify the memory, but you can't do this on Ubuntu 17.04 without recompiling the kernel as it requires: CONFIG_STRICT_DEVMEM=n, see also: How to access physical addresses from user space in Linux? Buildroot is an easy way to overcome that however.

Alternatively, you can use a Virtual machine like QEMU monitor's xp command: How to decode /proc/pid/pagemap entries in Linux?

See this to dump all pages: How to decode /proc/pid/pagemap entries in Linux?

Userland subset of this question: How to find the physical address of a variable from user-space in Linux?

Dump all process pages with /proc/<pid>/maps

/proc/<pid>/maps lists all the addresses ranges of the process, so we can walk that to translate all pages: /proc/[pid]/pagemaps and /proc/[pid]/maps | linux

Kerneland virt_to_phys() only works for kmalloc() addresses

From a kernel module, virt_to_phys(), has been mentioned.

However, it is import to highlight that it has this limitation.

E.g. it fails for module variables. arc/x86/include/asm/io.h documentation:

The returned physical address is the physical (CPU) mapping for the memory address given. It is only valid to use this function on addresses directly mapped or allocated via kmalloc().

Here is a kernel module that illustrates that together with an userland test.

So this is not a very general possibility. See: How to get the physical address from the logical one in a Linux kernel module? for kernel module methods exclusively.

7
  • -1. I tried this. In some cases it has spurious trailing zeros which makes the answer wrong. Here is a correct program. May 3, 2020 at 13:25
  • @AneesAhmed777 thanks for the comment, if you can give an example of when those trailing zeroes happen and what they look like, that would also be great. May 3, 2020 at 13:33
  • 2
    😅 Oops. I mistakenly thought the program is supposed to output only the PFN, but it actually outputs the whole physical address. The program works correctly: +1 upvoted. May 11, 2020 at 21:47
  • 2
    @AneesAhmed777 ah, thanks for confirming. Don't hesitate to tell me if you find anything that seems wrong. May 11, 2020 at 22:28
  • 1
    Weird I have r permission to pagemap as a regular user, are you aware of a sysctl config that can restrict the access to root-only? EDIT: It seems that I can only access pagemap of processes that I own.
    – vmemmap
    May 20, 2023 at 7:31
19

As answered before, normal programs should not need to worry about physical addresses as they run in a virtual address space with all its conveniences. Furthermore, not every virtual address has a physical address, the may belong to mapped files or swapped pages. However, sometimes it may be interesting to see this mapping, even in userland.

For this purpose, the Linux kernel exposes its mapping to userland through a set of files in the /proc. The documentation can be found here. Short summary:

  1. /proc/$pid/maps provides a list of mappings of virtual addresses together with additional information, such as the corresponding file for mapped files.
  2. /proc/$pid/pagemap provides more information about each mapped page, including the physical address if it exists.

This website provides a C program that dumps the mappings of all running processes using this interface and an explanation of what it does.

9

The suggested C program above usually works, but it can return misleading results in (at least) two ways:

  1. The page is not present (but the virtual addressed is mapped to a page!). This happens due to lazy mapping by the OS: it maps addresses only when they are actually accessed.
  2. The returned PFN points to some possibly temporary physical page which could be changed soon after due to copy-on-write. For example: for memory mapped files, the PFN can point to the read-only copy. For anonymous mappings, the PFN of all pages in the mapping could be one specific read-only page full of 0s (from which all anonymous pages spawn when written to).

Bottom line is, to ensure a more reliable result: for read-only mappings, read from every page at least once before querying its PFN. For write-enabled pages, write into every page at least once before querying its PFN.

Of course, theoretically, even after obtaining a "stable" PFN, the mappings could always change arbitrarily at runtime (for example when moving pages into and out of swap) and should not be relied upon.

5

I wonder why there is no user-land API.

Because user land memory's physical address is unknown.

Linux uses demand paging for user land memory. Your user land object will not have physical memory until it is accessed. When the system is short of memory, your user land object may be swapped out and lose physical memory unless the page is locked for the process. When you access the object again, it is swapped in and given physical memory, but it is likely different physical memory from the previous one. You may take a snapshot of page mapping, but it is not guaranteed to be the same in the next moment.

So, looking for the physical address of a user land object is usually meaningless.

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.