Vertica Blog

What Should I do if the Ancient History Mark is not Advancing?

Written by

If the AHM is not advancing, troubleshoot using this checklist:

Step Task Results
1 Check whether the Last Good Epoch (LGE) is advancing. => SELECT CURRENT_EPOCH, LAST_GOOD_EPOCH, AHM_EPOCH FROM SYSTEM; If the LGE is advancing, go to Step 2.

If the LGE is not advancing, go to Step 5.
2 Check whether all nodes are UP. => SELECT * FROM NODES WHERE NODE_STATE = 'UP'; If all the nodes are UP, go to Step 3. If one or more node is DOWN, bring all the nodes UP. $ admintools -t restart_node -d <database name> -s <node_name> Then proceed to Step 4.
3 Check for unrefreshed projections. => SELECT PROJECTION_NAME, NODE_NAME, IS_UP_TO_DATE FROM PROJECTIONS WHERE IS_UP_TO_DATE != 't'; If one or more of your projections is out of date (IS_UP_TO_DATE is f), do one of the following tasks:
  • Drop a projection => SELECT DROP PROJECTION <name of the projection>
  • Refresh a projection => SELECT REFRESH <table name>
4 Did the AHM advance? => SELECT GET_AHM_EPOCH(); If yes, this is the end of your checklist.
If no, start this checklist again to find an alternative path. If the AHM continues not to advance, contact Vertica Support.
5 Check whether there is data in WOS. => SELECT sum (wos_used_bytes) from projection_storage; If there is data in WOS, force a MOVEOUT. => SELECT DO_TM_TASK('moveout'); If there is no data in WOS, go to Step 6.
6 Check for a node where LGE is behind others. => SELECT GET_EXPECTED_RECOVERY_EPOCH(); If one node’s LGE is older than another node’s LGE, go to Step 7.

If all the nodes have the same LGE, open a ticket with Vertica Technical Support.
7 Check whether moveout is executing and move-out is not slow due to replay-deletes. => SELECT OPERATION_START_TIMESTAMP, NODE_NAME, OPERATION_NAME, OPERATION_STATUS, TABLE_SCHEMA, PROJECTION_NAME FROM TUPLE_MOVER_OPERATIONS WHERE IS_EXECUTING; If moveout is executing and the AHM advanced => SELECT GET_AHM_EPOCH(); This is end of your checklist. If moveout is not executing, open a ticket with Vertica Technical Support.

Learn More

Learn more about Epochs in the Vertica Documentation.