Knowledgebase: Index related
P5 Index Cleanup
Posted by Sven Koester, Last modified by Sven Koester on 15 March 2021 11:27
P5 indexes consist (among further info) of two parts:
  • The file and folder structure and change thereof over time, and
  • The media (tape) related tables with positions where the data is stored.
Both parts are independent from each other. In case for instance a file is saved on three tapes, it has three database records in the media part but just one in the file structure.

When deleting or relabeling a volume, the files of that volume can no longer be restored. In the index, they will however still be visible:
On deletion of a media (tape), the media part of the file will be marked as deleted. However, the file part is not, as that would require a check for each file whether it is on another tape. As such a test takes time, it is not done immediately.

When browsing in the Restore areas in P5, only the file and folder structure is regarded to allow fast browsing. In case the media part of the index would be tested, too, browsing would be rather slow. So it is possible that files which cannot be restored anymore are still visible in the index. They will remain visible until the index has been cleaned up.

It is always possible to check whether or not a file can be restored by opening the versions windows from the context menu. That window shows all media where the file is stored on and in which version.

Note that this is handled differently for files and folders: even if there are "versions" of of folder in the index, these are not exposed as they refer to the time stamps and other attributes of the folder, but not to its contents. A folder with a time
stamp from May can contain files that are much newer. So the "version" of a folder would be misleading.
Some folders may even appear without time stamp, these were not saved but exist only as nodes (to navigate to saved files and folders).

To figure out whether a folder contains files (without navigating down the folder tree), one can select the folder and click "restore to". P5 will then count the files and folders below and sum up the size before the actual restore is started. The restore can be cancelled before files are really restored, but the window shows size and number of the contained files.

In Backup indexes, the cleanup happens automatically after the next backup job using that index. The cleanup will appear in the index tables when 10% of the contained data is invalidated. So it may appear that if only a few files are affected, the cleanup is delayed.

In Archive indexes, there is no automated cleanup. If required it is possible to execute the cleanup manually.
The cleanup can be called through the nsdchat utility with the following commands:    
This call does the cleanup of elements no longer on tape.
Please make sure that during the cleanup, no archive or
restore jobs are running that use this index.

  /usr/local/aw/bin/nsdchat -c cli::index::purge Default-Archive true

(0 vote(s))
Not helpful

Comments (4)
Marc Levy
14 July 2019 21:52
this is very helpful but can you confirm the clean up does not require to read volumes?
i'm using cloud archive pools and will be charged if volumes are read upon this check.
Andre Kuehnemund
06 August 2019 20:52
P5 does not read the contents of the volumes themselves when cleaning up the index.
01 April 2021 16:38
how can we delete or edit entries from the archiware P5 database. Is there any possibility that we could delete entries from database of damage tapes or tapes would be recycle...?
Andre Kuehnemund
22 June 2021 23:36
@McKenzie: There are two ways to purge the respective index database of information that is no longer needed:

1. Relabel the tape or disk volume. Doing so will purge any file/folders records pertaining to that index from the database.

2. Select, then delete the respective tape or disk volume from the Volumes table in P5. This will also result in P5 removing all file/folder records pertaining to that index from the database.
Post a new comment
Full Name:
CAPTCHA Verification 
Please enter the text you see in the image into the textbox below (we use this to prevent automated submissions).