Techniques including a memory controller with a set of memory channel queues, wherein memory channel queues of the set of memory channel queues correspond to memory channels to access a set of memory modules, a first arbitration module, and a second arbitration module. The memory controller is configured to receive a first memory request from the peripheral and place one or more portions of the first memory request in the memory channel queues of the set of memory channel queues. The first arbitration module is configured to determine an arbitration algorithm, select a first memory channel queue based on the arbitration algorithm, present the one or more portions of the first memory request in the selected first memory channel queue to the second arbitration module, and output the presented one or more portions of the first memory request based on a selection by the second arbitration module.

According to one embodiment, a method for traffic prioritization in a memory device includes sending a memory access request including a priority value from a processing element in the memory device to a crossbar interconnect in the memory device. The memory access request is routed through the crossbar interconnect to a memory controller in the memory device associated with the memory access request. The memory access request is received at the memory controller. The priority value of the memory access request is compared to priority values of a plurality of memory access requests stored in a queue of the memory controller to determine a highest priority memory access request. A next memory access request is performed by the memory controller based on the highest priority memory access request.

Methods, systems, and apparatus, including computer-readable media, are described for interleaving memory requests to accelerate memory accesses at a hardware circuit configured to implement a neural network model. A system generates multiple requests that are processed against a memory of the system. Each request is used to retrieve data from the memory. For each request, the system generates multiple sub-requests based on a respective size of the data to be retrieved using the request. The system generates a sequence of interleaved sub-requests that includes respective sub-requests of a first request interleaved among respective sub-requests of a second request. Based on the sequence of interleaved sub-requests, a module of the system receives respective portions of data accessed from different address locations of the memory. The system processes each of the respective portions of data to generate a neural network inference using the neural network model implemented at the hardware circuit.

In one form, a memory controller includes a command queue, an arbiter, and a replay queue. The command queue receives and stores memory access requests. The arbiter is coupled to the command queue for providing a sequence of memory commands to a memory channel. The replay queue stores the sequence of memory commands to the memory channel, and continues to store memory access commands that have not yet received responses from the memory channel. When a response indicates a completion of a corresponding memory command without any error, the replay queue removes the corresponding memory command without taking further action. When a response indicates a completion of the corresponding memory command with an error, the replay queue replays at least the corresponding memory command. In another form, a data processing system includes the memory controller, a memory accessing agent, and a memory system to which the memory controller is coupled.

A memory controller includes an arbiter for selecting memory requests from a command queue for transmission to a dynamic random access memory (DRAM) memory. The arbiter includes a bank group tracking circuit that tracks bank group numbers of three or more prior write requests selected by the arbiter. The arbiter also includes a selection circuit that selects requests to be issued from the command queue, and prevents selection of write requests and associated activate commands to the tracked bank group numbers unless no other write request is eligible in the command queue. The bank group tracking circuit indicates that a prior write request and the associated activate commands are eligible to be issued after a number of clock cycles has passed corresponding to a minimum write-to-write timing period for a bank group of the prior write request.

A controller for controlling memory devices is provided to include: a first core configured to control first memory devices in communication with the controller and configured to store data associated with first logical addresses; a second core configured to control second memory devices in communication with the controller and configured to store data associated with second logical addresses; and a host interface configured to (1) queue commands received from a host in a queue, (2) perform a command reordering that determines a processing order of queued commands including a first address command associated with a first logical address and a second address command associated with a second logical address based on statuses of the first memory devices and the second memory devices, and (3) provide the first address command to the first core and the second address command to the second core based on the processing order.

Embodiments for performing cooperative user mode scheduling between user mode schedulable (UMS) threads and primary threads are disclosed. In accordance with one embodiment, privileged hardware states are transferred from a kernel portion of a UMS thread to a kernel portion of a primary thread.

Systems and methods are directed to managing access to a shared memory. A request received at a memory controller, for access to the shared memory from a client of one or more clients configured to access the shared memory, is placed in at least one queue in the memory controller. A series of one or more timeout values is assigned to the request, based, at least in part on a priority associated with the client which generated the request. The priority may be fixed or based on a Quality-of-Service (QoS) class of the client. A timer is incremented while the request remains in the first queue. As the timer traverses each one of the one or more timeout values in the series, a criticality level of the request is incremented. A request with a higher criticality level may be prioritized for servicing over a request with a lower criticality level.

Embodiments include method, systems and computer program products for a parallel ordering queue using an encoded command type. In some embodiments, a command may be receive from a receiver of a first bus, wherein the command is to be sent to a second bus. The command may be decoded. The command may be associated with an encoded command type. The command may be placed in an ordering queue. A first entry of a second queue may be popped based on the encoded command type of the first entry of the ordering queue. The first entry of the ordering queue may be removed from the ordering queue.

A data storage device includes a memory, a controller, and a communication bus coupled to the memory and to the controller. The controller is configured to send a read-write command and write data to the memory via the communication bus. The read-write command indicates an address of requested data to be read from the memory. The controller is further configured to receive the requested data read from the memory. Communicating the requested data over the communication bus overlaps the write data being stored into the memory.