This is the multi-page printable view of this section. Click here to print.

Return to the regular view of this page.

Reference

Reference

The diagram below illustrates the suggested order for reading the documentation:

graph
    Session
    Memory

    Session --> Memory
    Memory --> Objects
    Objects --> Buffer
    Objects --> Image
    Image --> ImageView

    Session --> NodeSystem
    NodeSystem --> ComputeNode
    ComputeNode --> ContainerNode

    %% Interaction
    click Session "/docs/reference/session" "Session"
    click Memory "/docs/reference/memory" "Memory"
    click Objects "/docs/reference/objects" "Objects"
    click Buffer "/docs/reference/objects/buffer" "Buffer"
    click Image "/docs/reference/image" "Image"
    click ImageView "/docs/reference/image_view" "ImageView"
    click NodeSystem "/docs/reference/node_system" "NodeSystem"
    click ComputeNode "/docs/reference/node_system/compute_node" "ComputeNode"
    click ContainerNode "/docs/reference/node_system/container_node" "ContainerNode"

1 - Session

A Session is the main object in a Lluvia application. It holds the references to the underlying device used for computation. To see the available devices, run:

1
2
3
4

import lluvia as ll

for device in ll.getAvailableDevices():
    print(device)
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17

#include <lluvia/core.h>

#include <iostream>

int main() {

    const auto availableDevices = ll::Session::getAvailableDevices();

    for(auto deviceDesc : availableDevices) {

        std::cout << "ID: " << deviceDesc.id
                  << " type: " << ll::deviceTypeToString(std::forward<ll::DeviceType>(deviceDesc.deviceType))
                  << " name: " << deviceDesc.name << std::endl;
    }

    return 0;
}

and the output can look like:

id: 7040 type: DiscreteGPU   name: GeForce GTX 1080
id: 0    type: CPU           name: llvmpipe (LLVM 12.0.0, 256 bits)
id: 1042 type: IntegratedGPU name: Intel(R) HD Graphics 4600 (HSW GT2)

To create a session:

1
2
3
4
5
6
7
8

import lluvia as ll

devices = ll.getAvailableDevices()

# ... select the device appropriate to your needs
selectedDevice = devices[0]

session = ll.createSession(enableDebug=True, device=selectedDevice)
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19

#include <lluvia/core.h>

#include <memory>

int main() {

    const auto availableDevices = ll::Session::getAvailableDevices();

    // ... select the device appropriate to your needs
    auto selectedDevice = availableDevices[0];

    auto desc = ll::SessionDescriptor()
                    .enableDebug(true)
                    .setDeviceDescriptor(selectedDevice);    

    std::shared_ptr<ll::Session> session = ll::Session::create(desc);

    return 0;
}

The enableDebug flag enables the Vulkan validation layers for receiving messages about bad usage of the API. This can be useful while building your compute pipelines, but should be disabled in Production for reducing the communication overhead with the GPU.

Several object types are creating from a session, among the most important are:

graph
    Session --> Memory
    Session --> Program
    Session --> CommandBuffer
    Session --> Duration
    Session --> ComputeNode
    Session --> ContainerNode

What’s next

Check the Memory page to know about the different memory types in Lluvia.

2 - Memory

Memory objects represent regions of memory that can be used to allocate objects. Lluvia uses the memory types defined by the Vulkan API. You may also refer to this article by Adam Sawicki on how memory is offered by different GPU vendors.

Memory types

Memory objects are created from a Lluvia Session. The code block below enumerate the available memory options:

1
2
3
4
5
6
7
8

import lluvia as ll

session = ll.createSession(enableDebug=True)

for n, memflags in enumerate(session.getSupportedMemoryPropertyFlags()):
    print('Memory index:', n)
    print('    supported flags:', [p.name for p in memflags])
    print()
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22

#include <iostream>
#include "lluvia/core.h"

#include <vulkan/vulkan.hpp>

int main() {
    
    ll::SessionDescriptor desc = ll::SessionDescriptor().enableDebug(true);

    std::shared_ptr<ll::Session> session = ll::Session::create(desc);

    std::vector<vk::MemoryPropertyFlags> flagsVector = session->getSupportedMemoryFlags();

    for(int i = 0; i < flagsVector.size(); ++i) {
        const vk::MemoryPropertyFlags &flags = flagsVector[i];
        
        std::cout << "Memory index: " << i << std::endl;
        std::cout << "    Supported flags: " << vk::to_string(flags) << std::endl;
    }

    return 0;
}

The possible MemoryPropertyFlags values are:

FlagDescription
DeviceLocalThe memory is visible to the GPU.
HostVisibleThe memory is visible to the host (CPU).
HostCoherentIf set, it indicates that read/write operations on the memory are coherent. That is, no flushing is needed for making the values visible by other consumers.
HostCachedIf set, it indiates that read/write operations travel through the host memory cache. Operations may be faster, but need flushing to make the values available to other consumers.
LazilyAllocatedNot used in Lluvia.

For Lluvia, the two most important memory flag tuples are:

TupleDescription
(DeviceLocal)The memory is visible to the GPU only. Computations will be performed on objects allocated in this memory.
(DeviceLocal, HostVisible, HostCoherent)The memory is visible to both the GPU and the host CPU. Writings to the memory from the host CPU are coherent. This memory will be used mainly for transfering data to and from the GPU.

Creation

The code block below shows how to create memory objects:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10

import lluvia as ll

session = ll.createSession(enableDebug=True)

deviceMemory = session.createMemory(ll.MemoryPropertyFlagBits.DeviceLocal, pageSize=32*1024*1024, exactFlagsMatch=False)

# use default page size
hostMemory = session.createMemory([ll.MemoryPropertyFlagBits.DeviceLocal,
                                   ll.MemoryPropertyFlagBits.HostVisible,
                                   ll.MemoryPropertyFlagBits.HostCoherent])
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20

#include "lluvia/core.h"
#include <vulkan/vulkan.hpp>
#include <iostream>
#include <memory>

int main() {

    auto session = ll::Session::create(ll::SessionDescriptor().enableDebug(true));
    
    const vk::MemoryPropertyFlags deviceFlags = vk::MemoryPropertyFlagBits::eDeviceLocal;
    const vk::MemoryPropertyFlags hostFlags = vk::MemoryPropertyFlagBits::eDeviceLocal |
                                              vk::MemoryPropertyFlagBits::eHostVisible |
                                              vk::MemoryPropertyFlagBits::eHostCoherent;

    std::shared_ptr<ll::Memory> deviceMemory = session->createMemory(deviceFlags, 32*1024*1024, false);

    std::shared_ptr<ll::Memory> hostMemory = session->createMemory(hostFlags, 32*1024*1024, false);

    return 0;
}

Memories are created by passing the set of flags the memory should have. It is possible to also pass the size of a page, which defaults to 32MB, and an extra parameter to indicate if the flags should match perfectly with any of those listed by session.getSupportedMemoryPropertyFlags().

Internally, a Memory manages regions of memory as pages. On each page, there can be several objects allocated, such as Buffer or Image.

stateDiagram-v2

    Memory --> Page_0
    Memory --> Page_1
    Memory --> Page_2

    state Page_0 {
        Buffer_0
        Buffer_1
    }

    state Page_1 {
        Image_1
        Buffer_2
    }

    state Page_2 {
        Image_2
    }

It is possible to query the memory attributes as:

1
2
3
4

print('flags      :', [p.name for p in hostMemory.memoryFlags])
print('isMappable :', hostMemory.isMappable)
print('pageCount  :', hostMemory.pageCount)
print('pageSize   :', hostMemory.pageSize)
1
2
3
4

std::cout << "flags      : " << vk::to_string(hostMemory->getMemoryPropertyFlags()) << std::endl;
std::cout << "isMappable : " << hostMemory->isMappable() << std::endl;
std::cout << "pageCount  : " << hostMemory->getPageCount() << std::endl;
std::cout << "pageSize   : " << hostMemory->getPageSize() << std::endl;

which prints:

In particular, the isMappable flag tells whether or not the memory space can be mapped to the host memory space. At the moment of creation, there are no actual pages allocated, and hence, pageCount equals 0.

Object allocation

There are two types of objects that can be allocated from a Memory:

graph
    Memory --> Buffer
    Memory --> Image

    click Buffer "/docs/reference/objects/buffer" "Buffer"
    click Image "/docs/reference/image" "Image"

The code block below shows how to allocate a buffer and an image object. Each allocated object has an allocationInfo to see the allocation values.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24

import lluvia as ll

session = ll.createSession(enableDebug=True)

deviceMemory = session.createMemory(ll.MemoryPropertyFlagBits.DeviceLocal, pageSize=32*1024*1024)

# A 1024 byte size buffer
buffer = deviceMemory.createBuffer(1024)

# A 32x32 pixels image where each pixel is of type Uint8
image = deviceMemory.createImage((32, 32), ll.ChannelType.Uint8)

print('buffer:')
print('  page         :', buffer.allocationInfo.page)
print('  offset       :', buffer.allocationInfo.offset)
print('  left padding :', buffer.allocationInfo.leftPadding)
print('  size         :', buffer.allocationInfo.size)

print()
print('image:')
print('  page         :', image.allocationInfo.page)
print('  offset       :', image.allocationInfo.offset)
print('  left padding :', image.allocationInfo.leftPadding)
print('  size         :', image.allocationInfo.size)
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31

#include "lluvia/core.h"
#include <vulkan/vulkan.hpp>
#include <iostream>
#include <memory>

int main() {
    auto session = ll::Session::create(ll::SessionDescriptor().enableDebug(true));
    
    const vk::MemoryPropertyFlags deviceFlags = vk::MemoryPropertyFlagBits::eDeviceLocal;

    std::shared_ptr<ll::Memory> deviceMemory = session->createMemory(deviceFlags, 32*1024*1024, false);
    
    std::shared_ptr<ll::Buffer> buffer = deviceMemory->createBuffer(1024);
    
    // 32x32 image with one uint8 color channel per pixel
    ll::ImageDescriptor desc = ll::ImageDescriptor(1, 32, 32, ll::ChannelCount::C1, ll::ChannelType::Uint8);
    std::shared_ptr<ll::Image> image = deviceMemory->createImage(desc);
    
    std::cout << "buffer:" << std::endl;
    std::cout << "  page         : " << buffer->getAllocationInfo().page << std::endl;
    std::cout << "  offset       : " << buffer->getAllocationInfo().offset << std::endl;
    std::cout << "  left padding : " << buffer->getAllocationInfo().leftPadding << std::endl;
    std::cout << "  size         : " << buffer->getAllocationInfo().size << std::endl;

    std::cout << std::endl;
    std::cout << "image:" << std::endl;
    std::cout << "  page         : " << image->getAllocationInfo().page << std::endl;
    std::cout << "  offset       : " << image->getAllocationInfo().offset << std::endl;
    std::cout << "  left padding : " << image->getAllocationInfo().leftPadding << std::endl;
    std::cout << "  size         : " << image->getAllocationInfo().size << std::endl;
}

The amount of memory reserved for a given object can be higher than the actual needed. This is because Vulkan imposes certain requirements on the allocation such as alignment.

What’s next

Check the Objects page for an overview of the objects available in Lluvia.

3 - Objects

3.1 - Buffer

Buffers are unstructured regions of contiguous memory. Buffers are created from Memory objects:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11

import lluvia as ll

session = ll.createSession()

hostMemory = session.createMemory([ll.MemoryPropertyFlagBits.DeviceLocal,
                                   ll.MemoryPropertyFlagBits.HostVisible,
                                   ll.MemoryPropertyFlagBits.HostCoherent])

aBuffer = hostMemory.createBuffer(1024, usageFlags=[ll.BufferUsageFlagBits.TransferDst,
                                                    ll.BufferUsageFlagBits.TransferSrc,
                                                    ll.BufferUsageFlagBits.StorageBuffer])
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22

#include <iostream>
#include <vulkan/vulkan.hpp>
#include "lluvia/core.h"

#include <vulkan/vulkan.hpp>

int main() {
    
    ll::SessionDescriptor desc = ll::SessionDescriptor().enableDebug(true);

    std::shared_ptr<ll::Session> session = ll::Session::create(desc);

    hostMemory = session.createMemory([ll.MemoryPropertyFlagBits.DeviceLocal,
                                   ll.MemoryPropertyFlagBits.HostVisible,
                                   ll.MemoryPropertyFlagBits.HostCoherent])

    const auto usageFlags = vk::BufferUsageFlags { vk::BufferUsageFlagBits::eStorageBuffer
                                                 | vk::BufferUsageFlagBits::eTransferSrc
                                                 | vk::BufferUsageFlagBits::eTransferDst};

    auto aBuffer = hostMemory->createBuffer(1024, usageFlags);
}

The first parameter is the requested size in bytes. The usageFlags indicated the intended usage of this buffer; the values are taken directly from the Vulkan BufferUsageFlagBits. The most used values are:

FlagDescription
StorageBufferIndicates that the buffer is going to be used for general storage.
TransferDstIndicates that the buffer can be used as destination for transfer commands.
TransferSrcIndicates that the buffer can be used as source for transfer commands.

3.2 - Image

3.3 - ImageView

4 - Node system

%%{init: {'theme': 'neutral', 'themeVariables': {'fontSize': '32px', 'primaryColor': '#FF0000'}}}%%

classDiagram

    class Session
    class Memory
    class Program
    
    class Buffer
    class Image
    class ImageView

    class Node
    class ComputeNode
    class ContainerNode

    class CommandBuffer

    Session "1" --> "*" Memory
    Session "1" --> "*" Node
    Session "1" --> "*" CommandBuffer

    Memory "1" --> "*" Buffer: allocates
    Memory "1" --> "*" Image: allocates
    Image "1" --> "*" ImageView: creates

    Node <|-- ComputeNode
    Node <|-- ContainerNode

    ComputeNode "1" --> "1" Program

    ContainerNode "1" --> "*" ComputeNode: contains
    ContainerNode "1" --> "*" ContainerNode: contains

4.1 - Compute nodes

4.2 - Container nodes

4.3 - Running