Method, device and computer program product for managing metadata at a control device
10936489 ยท 2021-03-02
Assignee
Inventors
- Geng Han (Beijing, CN)
- Yousheng Liu (Beijing, CN)
- Xiongcheng Li (Beijing, CN)
- Lifeng Yang (Beijing, CN)
- Jianbin Kang (Beijing, CN)
Cpc classification
G06F12/0868
PHYSICS
G06F2212/7201
PHYSICS
G06F2212/1032
PHYSICS
International classification
Abstract
Techniques for managing metadata at a control device involve: determining, from a cache page corresponding to user data, a first region for storing raw metadata of the user data, the raw metadata including address information for storing the user data in a storage system; in response to the user data being modified, determining updated metadata of the modified user data to update the raw metadata in the first region; and copying the updated metadata to a high-speed memory shared by the control device and another control device. Accordingly, the techniques are capable of reducing the usage frequency of the high-speed memory, thereby extending the service life of the high-speed memory and reducing cost.
Claims
1. A method for managing metadata at a control device, comprising: determining, from a cache page corresponding to user data, a first region for storing raw metadata of the user data, the raw metadata comprising address information for storing the user data in a storage system; in response to the user data being modified, determining updated metadata of the modified user data to update the raw metadata in the first region; copying the updated metadata to a high-speed memory shared by the control device and another control device; in response to metadata stored on a second region of the cache page being updated, merging the first region and the second region into a third region; and copying the metadata stored in the third region to the shared high-speed memory; wherein merging the first region and the second region into a third region comprises: determining a first starting position and a first length of the first region in the cache page; determining a second starting position and a second length of the second region in the cache page; determining a starting position of the third region based on the first starting position and the second starting position; and determining a length of the third region, based on the first starting position, the first length, the second starting position, and the second length.
2. The method according to claim 1, wherein determining the first region comprises: determining the first region in the cache page for storing the raw metadata, based on description information of the cache page.
3. The method according to claim 1, wherein determining updated metadata of the modified user data comprises: determining, based on address mapping management information, address information of the modified user data stored in the storage system; and generating the updated metadata based on the determined address information.
4. The method according to claim 1, further comprising: sending an update message to the another control device to inform the another control device to use the updated metadata that has been stored in the shared high-speed memory.
5. The method according to claim 1, further comprising: in response to receiving an access request for the user data, redirecting the access request to one of the control device and the another control device based on address information of the user data to be accessed.
6. A control device for managing metadata, comprising: at least one processing unit; at least one memory coupled to the at least one processing unit and storing instructions executed by the at least one processing unit, the instructions, when executed by the at least one processing unit, causing the control device to perform actions comprising: determining, from a cache page corresponding to user data, a first region for storing raw metadata of the user data, the raw metadata including address information for storing the user data in a storage system; in response to the user data being modified, determining updated metadata of the modified user data to update the raw metadata in the first region; copying the updated metadata to a high-speed memory shared by the control device and another control device; in response to metadata stored on a second region of the cache page being updated, merging the first region and the second region into a third region; and copying the metadata stored in the third region to the shared high-speed memory; wherein merging the first region and the second region into a third region comprises: determining a first starting position and a first length of the first region in the cache page; determining a second starting position and a second length of the second region in the cache page; determining a starting position of the third region based on the first starting position and the second starting position; and determining a length of the third region, based on the first starting position, the first length, the second starting position, and the second length.
7. The control device according to claim 6, wherein determining the first region comprises: determining the first region in the cache page for storing the raw metadata, based on description information of the cache page.
8. The control device according to claim 6, wherein determining updated metadata of the modified user data comprises: determining, based on address mapping management information, address information of the modified user data stored in the storage system; and generating the updated metadata based on the determined address information.
9. The control device according to claim 6, wherein the actions further comprise: sending an update message to the another control device to inform the another control device to use the updated metadata that has been stored in the shared high-speed memory.
10. The control device according to claim 6, wherein the actions further comprise: in response to receiving an access request for the user data, redirecting the access request to one of the control device and the another control device based on address information of the user data to be accessed.
11. A computer program product having a non-transitory computer readable medium which stores a set of instructions to manage metadata at a control device; the set of instructions, when carried out by computerized circuitry, causing the computerized circuitry to perform a method of: determining, from a cache page corresponding to user data, a first region for storing raw metadata of the user data, the raw metadata comprising address information for storing the user data in a storage system; in response to the user data being modified, determining updated metadata of the modified user data to update the raw metadata in the first region; copying the updated metadata to a high-speed memory shared by the control device and another control device; in response to metadata stored on a second region of the cache page being updated, merging the first region and the second region into a third region; and copying the metadata stored in the third region to the shared high-speed memory; wherein merging the first region and the second region into a third region comprises: determining a first starting position and a first length of the first region in the cache page; determining a second starting position and a second length of the second region in the cache page; determining a starting position of the third region based on the first starting position and the second starting position; and determining a length of the third region, based on the first starting position, the first length, the second starting position, and the second length.
12. The computer program product according to claim 11, wherein determining the first region comprises: determining the first region in the cache page for storing the raw metadata, based on description information of the cache page.
13. The computer program product according to claim 11, wherein determining updated metadata of the modified user data comprises: determining, based on address mapping management information, address information of the modified user data stored in the storage system; and generating the updated metadata based on the determined address information.
14. The computer program product according to claim 11, wherein the method further includes: sending an update message to the another control device to inform the another control device to use the updated metadata that has been stored in the shared high-speed memory.
15. The computer program product according to claim 11, wherein the method further includes: in response to receiving an access request for the user data, redirecting the access request to one of the control device and the another control device based on address information of the user data to be accessed.
16. A storage system, comprising: control devices constructed and arranged to access a set of storage devices; and a high-speed memory coupled with the control devices; each control device including: at least one processing unit; at least one memory coupled to the at least one processing unit and storing instructions executed by the at least one processing unit, the instructions, when executed by the at least one processing unit, causing the control device to perform actions comprising: determining, from a cache page corresponding to user data, a first region for storing raw metadata of the user data, the raw metadata including address information for storing the user data; in response to the user data being modified, determining updated metadata of the modified user data to update the raw metadata in the first region; copying the updated metadata to the high-speed memory; in response to metadata stored on a second region of the cache page being updated, merging the first region and the second region into a third region; and copying the metadata stored in the third region to the high-speed memory; wherein merging the first region and the second region into a third region comprises: determining a first starting position and a first length of the first region in the cache page; determining a second starting position and a second length of the second region in the cache page; determining a starting position of the third region based on the first starting position and the second starting position; and determining a length of the third region, based on the first starting position, the first length, the second starting position, and the second length.
17. The storage system according to claim 16, wherein determining the first region comprises: determining the first region in the cache page for storing the raw metadata, based on description information of the cache page.
18. The storage system according to claim 16, wherein determining updated metadata of the modified user data comprises: determining, based on address mapping management information, address information of the modified user data stored in the storage system; and generating the updated metadata based on the determined address information.
19. The storage system according to claim 16, wherein the actions further comprise: sending an update message to another control device to inform the another control device to use the updated metadata that has been stored in the high-speed memory.
20. The storage system according to claim 16, wherein the actions further comprise: in response to receiving an access request for the user data, redirecting the access request to another control device based on address information of the user data to be accessed.
Description
BRIEF DESCRIPTION OF THE DRAWINGS
(1) Through the following detailed description with reference to the accompanying drawings, the above and other objectives, features, and advantages of example embodiments of the present disclosure will become more apparent. In example embodiments of the present disclosure, the same reference symbols usually refer to the same components.
(2)
(3)
(4)
(5)
(6)
(7)
(8)
(9)
(10)
(11) In the figures, the same or corresponding reference numbers represent the same or corresponding parts.
DETAILED DESCRIPTION OF EMBODIMENTS
(12) The individual features of the various embodiments, examples, and implementations disclosed within this document can be combined in any desired manner that makes technological sense. Furthermore, the individual features are hereby combined in this manner to form all possible combinations, permutations and variants except to the extent that such combinations, permutations and/or variants have been explicitly excluded or are impractical. Support for such combinations, permutations and variants is considered to exist within this document.
(13) It should be understood that the specialized circuitry that performs one or more of the various operations disclosed herein may be formed by one or more processors operating in accordance with specialized instructions persistently stored in memory. Such components may be arranged in a variety of ways such as tightly coupled with each other (e.g., where the components electronically communicate over a computer bus), distributed among different locations (e.g., where the components electronically communicate over a computer network), combinations thereof, and so on.
(14) Preferred embodiments of the present disclosure will be described in greater detail with reference to the drawings. Although the drawings present the preferred embodiments of the present disclosure, it should be understood that the present disclosure can be implemented in various ways and should not be limited by the embodiments disclosed herein. Rather, those embodiments are provided for thorough and complete understanding of the present disclosure, and completely conveying the scope of the present disclosure to those skilled in the art.
(15) The term comprise and its variations used in the present disclosure mean comprising in an open-ended sense, i.e. comprise without limitation. Unless otherwise specified, the term or means and/or. The term based on means at least partially based on. The terms one exemplary embodiment and one embodiment represent at least one embodiment; the term another embodiment represents at least one another embodiment. The terms first, second and the like may refer to different or the same objects. Other explicit and implicit definitions might further be comprised in the following description.
(16)
(17) As shown in
(18) In
(19) The first control device 120 and the second control device 150 share a high-speed memory 140, such as NVMe. The high-speed memory 140 may store data shared by the first control device 120 and the second control device 150, such as metadata or the like.
(20)
(21)
(22) Each user data has an address ID, which includes a set of addresses. Each address in the set of addresses describes a corresponding entry in each level in the tree structure, thereby implementing mapping from a logical address to a virtual addresses.
(23) In the example of
(24) Usually, when the user performs an operation such as modification for the user data in the user data cache, the metadata such as the address information corresponding to the user data is also changed. The changed metadata is copied to the high-speed memory 140 shared by the first control device 120 and the second control device 150 before being saved to the storage device 110.
(25) Conventionally, when the metadata of the user data, in particular, the address information changes, all the contents of the cache page in which the changed metadata is located are copied into the high-speed memory 140, thereby causing data to be frequently written into the high-speed memory 140. This causes the high-speed memory to age too quickly, thereby reducing the service life, and increasing the cost of the storage system.
(26) In order to at least partially solve the above problems and potential other problems, embodiments of the present disclosure provide a concept for managing metadata at a control device.
(27)
(28) At 410, a first region of raw metadata for storing user data is determined from a cache page corresponding to user data. The raw metadata includes address information in the storage system that is used to store the user data. In some embodiments, the raw metadata may be metadata in a cache page for storing user data that has not been modified. The address information may be, for example, a starting address, an ending address, an address length, a combination thereof, and the like.
(29) In some embodiments, the first region in the cache page for storing the raw metadata may be determined based on the description information of the cache page. This will be described in detail below in conjunction with the specific embodiments of
(30) At 420, it is determined whether the user data has been modified. If yes, then at 430, updated metadata for the modified user data is determined to update the raw metadata in the first region. Here, the updated metadata refers to metadata for storing changed user data in a cache page due to a change in the user data.
(31) In some embodiments, the address information in which the modified user data is stored in the storage system may be determined based on address mapping management information. Then, the updated metadata may be generated based on the determined address information. This will be described in detail below in conjunction with the specific embodiments of
(32) Further referring to
(33) With the concept in method 400, only the modified metadata portion of the cache page is copied into the high-speed memory. Therefore, the concept can reduce the usage frequency of storage units in the high-speed memory, thereby extending the service life of the high-speed memory and reducing the cost.
(34) Additionally, in some embodiments, when the metadata on the second region of the cache page is updated, the first region and the second region are merged into a third region, and the metadata stored in the third region is copied to the shared high-speed memory. This will be described in more detail below in conjunction with
(35) Additionally, in some embodiments, upon receiving an access request for user data, the access request is redirected to one of the control device and another control device based on the address information of the user data to be accessed. This will be described in more detail below in conjunction with
(36)
(37) In
(38) In
(39) In an embodiment of the disclosure, in order to modify the metadata in a cache page, the following four application programming interfaces APIs can be used to perform the corresponding basic operations:
(40) MMDC_SET (element, val): set the element to value val;
(41) MMDC_ADD (element, val): add the element with value val;
(42) MMDC_MEM_COPY (dst_ptr, src_ptr, len): copy the data of length len from a region src in the cache page to a region dst;
(43) MMDC_MEM_SET (ptr, val, len): set the region ptr of length len in the cache page to value val.
(44) While the above four basic operations are performed for the metadata in the cache page to update the metadata, for example, the modified start position and length in the metadata may be recorded to determine the modified region in the metadata.
(45) In some embodiments, when performing the above four basic operations, description information for the cache page is modified, and a region for storing metadata of the modified user data in the cache page is determined based on the description information.
(46) The modification of the description information of the cache page can be implemented in various ways. In some embodiments, the following codes may be used to make the above modifications:
(47) TABLE-US-00001 #define MMDC_MARK_MODIFY(modify_addr, modify_len) \ { \ page_descriptor_t *pd; \ pd = get_pd(modify_addr); \ if (pd>modify_start_addr == NULL) \ { \ pd>modify_start_addr = modify_addr; \ pd>len = modify_len; \ } \ else \ { \ void *old_end_addr = pd>modify_start_addr + pd>len; \ if ((void*)modify_addr < pd>modify_start_addr) \ { \ pd>modify_start_addr = modify_addr; \ pd>len = old_end_addr pd>modify_start_addr; \ } \ if (old_end_addr < (void*)modify_addr + modify_len) \ } \ pd>len = (void*)modify_addr + modify_len \ pd>modify_start_addr; } \ } \ } \
(48) It should be understood that the above codes merely illustrates an example implementation. Those skilled in the art will appreciate that modifications to the description information of the cache page may also be implemented using other software forms, hardware forms, or a combination of software and hardware.
(49)
(50) At 610, it is determined whether there is a plurality of modified regions in the data stored on the cache page. This may also be determined, for example, by means of the description information of the cache page mentioned above. At 620, the modified first and second regions in the cache page are merged into a total third region. This merging process is described in more detail below in conjunction with
(51)
(52) In one embodiment of the disclosure, merging the first region 720 and the second region 730 into the third region 740 includes determining a first starting position and a first length of the first region 720 in the cache page; determining a second starting position and a second length of the second region 730 in the cache page; determining a starting position of the third region 740 based on the first starting position and the second starting position; and based on the first starting position, the first length, the second starting position, and the second length, determining the length of the third region 740.
(53) In some embodiments, a range of the third region may also be determined by means of the starting and ending positions, or the ending positions and lengths of the first and second regions.
(54) Further referring to
(55) In some embodiments, the storage system 100 may include a plurality of control devices, such as first control device 120 and second control device 150, to serve a user's data access request. The first control device 120 and the second control device 150 can share a high-speed memory. When the first control device 120 in the storage system 100 for example copies the updated metadata to the high-speed memory, to ensure that the second control device 150 can acquire the updated data, the first control device 120 sends an update message to the second control device 150 to inform the second control device to use the updated metadata that has been stored in the shared high-speed memory, namely, the second control device 150 retrieves the metadata in the modified region in the cache page from the shared high-speed memory 140.
(56) In some embodiments, the second control device 150 needs to retrieve from the storage device 110 raw metadata in the cache page corresponding to the modified user data. The updated metadata in the entire cache page is then derived based on the raw metadata and the modified metadata retrieved from the high-speed memory 140. Since the read speed of the storage device 110 is generally lower than the read speed of the high-speed memory 140, the above process may result in degradation of the performance of the storage system 100.
(57) In order to avoid frequent occurrence of the above process, in some embodiments according to the present disclosure, the access request is redirected to one of the first control device 120 and the second control device 150 based on the address information of the user data to be accessed by the data access request. This redirection process is described in more detail below through the embodiment of
(58)
(59) In this way, the frequency of the first control device 120 and the second control device 150 using the updated metadata of the modified user data stored in the shared high-speed memory may be reduced, thereby improving the performance of the entire storage system 100.
(60)
(61) A plurality of components in the device 900 are connected to the I/O interface 950, including: an input unit 960, such as keyboard, mouse and the like; an output unit 970, such as various types of display, loudspeakers and the like; a storage unit 980, such as magnetic disk, optical disk and the like; and a communication unit 990, such as network card, modem, wireless communication transceiver and the like. The communication unit 990 allows the device 900 to exchange information/data with other devices through computer networks such as Internet and/or various telecommunication networks.
(62) Each procedure and processing described above, such as method 400 and/or 600, may be executed by a processing unit 910. As an example, in some embodiments, the method 400 and/or 600 may be implemented as computer software programs, which are tangibly comprised in a machine-readable medium, such as storage unit 980. In some embodiments, the computer program may be partially or completely loaded and/or installed to the device 900 via ROM 920 and/or the communication unit 990. When the computer program is loaded to RAM 930 and executed by CPU 910, one or more acts of the above described method 400 and/or 600 are implemented.
(63) The present disclosure may be a method, a device, a system and/or a computer program product. The computer program product can include a computer-readable storage medium loaded with computer-readable program instructions thereon for executing various aspects of the present disclosure.
(64) The computer readable storage medium may be a tangible device capable of holding and storing instructions used by an instruction execution device. The computer readable storage medium may be, but is not limited to, for example, electronic storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any random appropriate combination thereof. More specific examples (non-exhaustive list) of the computer readable storage medium includes: a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), a static random access memory (SRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanically encoded device such as a punched card storing instructions or an emboss within a groove, and any random suitable combination thereof. A computer readable storage medium used herein is not interpreted as a transitory signals per se, such as radio waves or other freely propagated electromagnetic waves, electromagnetic waves propagated through a waveguide or other transmission medium (e.g., optical pulses passing through fiber-optic cables), or electrical signals transmitted through electric wires.
(65) The computer readable program instructions described herein may be downloaded from a computer readable storage medium to various computing/processing devices, or to external computers or external storage devices via a network, for example, the Internet, a local area network, a wide area network and/or a wireless network. The network may include copper transmission cables, optical transmission fibers, wireless transmission, routers, firewalls, switches, gateway computers and/or edge servers. The network adapter or network interface in each computing/processing device receives computer readable program instructions from the network and forwards the computer readable program instructions for storage in a computer readable storage medium of each computing/processing device.
(66) Computer readable program instructions for executing the operations of the present disclosure may be assembly instructions, instructions of instruction set architecture (ISA), machine instructions, machine dependent instructions, microcode, firmware instructions, state setting data, or either source code or destination code written by any combination of one or more programming languages including object oriented programming languages, such as Smalltalk, C++ or the like, and conventional procedural programming languages, such as the C progamming language or similar programming languages. The computer-readable program instructions may be completely or partially executed on the user computer, or executed as an independent software package, or executed partially on the user computer and partially on the remote computer, or completely executed on the remote computer or the server. In the case where a remote computer is involved, the remote computer may be connected to the user computer by any type of networks, including local area network (LAN) or wide area network (WAN), or connected to an external computer (such as via Internet provided by the Internet service provider). In some embodiments, the electronic circuit is customized by using the state information of the computer-readable program instructions. The electronic circuit may be a programmable logic circuit, a field programmable gate array (FPGA) or a programmable logic array (PLA) for example. The electronic circuit may execute computer-readable program instructions to implement various aspects of the present disclosure.
(67) Various aspects of the present disclosure are described in reference with the flow chart and/or block diagrams of method, device (systems) and computer program product according to embodiments of the present disclosure. It will be understood that each block in the flow chart and/or block diagrams, and any combinations of various blocks thereof may be implemented by computer readable program instructions.
(68) The computer-readable program instructions may be provided to the processing unit of a general purpose computer, a dedicated computer or other programmable data processing devices to generate a machine, causing the instructions, when executed by the processing unit of the computer or other programmable data processing devices, to generate a device for implementing the functions/actions specified in one or more blocks of the flow chart and/or block diagram. The computer-readable program instructions may also be stored in the computer-readable storage medium. These instructions enable the computer, the programmable data processing device and/or other devices to operate in a particular way, such that the computer-readable medium storing instructions may include a manufactured article that includes instructions for implementing various aspects of the functions/actions specified in one or more blocks of the flow chart and/or block diagram.
(69) The computer readable program instructions may also be loaded into computers, other programmable data processing devices, or other devices, so as to execute a series of operational steps on the computer, other programmable data processing devices or other devices to generate a computer implemented process. Therefore, the instructions executed on the computer, other programmable data processing devices, or other device may realize the functions/actions specified in one or more blocks of the flow chart and/or block diagram.
(70) The accompanying flow chart and block diagram present possible architecture, functions and operations realized by the system, method and computer program product according to a plurality of embodiments of the present disclosure. At this point, each block in the flow chart or block diagram may represent a module, a program segment, or a portion of the instruction. The module, the program segment or the portion of the instruction includes one or more executable instructions for implementing specified logic functions. In some alternative implementations, the function indicated in the block may also occur in an order different from the one represented in the drawings. For example, two consecutive blocks actually may be executed in parallel, and sometimes they may also be executed in a reverse order depending on the involved functions. It should also be noted that each block in the block diagram and/or flow chart, and any combinations of the blocks thereof may be implemented by a dedicated hardware-based system for implementing specified functions or actions, or a combination of the dedicated hardware and the computer instructions.
(71) Various embodiments of the present disclosure have been described above, and the above explanation is illustrative rather than exhaustive and is not limited to the disclosed embodiments. Without departing from the scope and spirit of each explained embodiment, many alterations and modifications are obvious for those ordinary skilled in the art. The selection of terms in the text aims to best explain principle, actual application or technical improvement in the market of each embodiment or make each embodiment disclosed in the text comprehensible for those ordinary skilled in the art.