Portal publish error- generated null data, column not nullable

Document ID : KB000008928
Last Modified Date : 14/02/2018
Show Technical Document Details
Issue:

The issue reported is the publishing works in Datamaker, but not in the Portal publish engine.  From the Addition Information table on the left of the Jobs table, the Output message states "Generated null data for table BEACONDAILYCOUNTS, column ACCOUNT_ID which is not nullable."  

However, the data does not contain any non null data. All Data Painter values evaluate to a non-nullable value as expected. This leaves the question in this case why is the publish failing? Where does one begin to look for the errors?

Environment:
TDM 4.3
Cause:

When publishing in the TDM Portal, the log to start looking for publish errors is the TDMPublish.log file: 

The KB referenced in this case was TDM Portal: How to enable debugging for TDM Portal

Our starting points for this case are the following:

1. During publish there are no rows found in the table dbo.Open_Orders , hence the exception was thrown. 
2. In the pre-publish action the data from dbo.Open_Orders table is deleted first and then some other actions are executed. 

The above was discovered while examining this section of the TDMPublish.log in this specific case:
c.c.t.p.m.TDMPublishJobStart: executeActions: doing action: TCORDER CLEARDOWN 
2017-12-06 21:47:21.599 UTC [DEBUG] [taskExecutor-3 ] --- [U:][M:][P:] c.c.t.s.g.MetaFunctions: uf_resolve_meta_data_item: evaluating expression: DELETE dbo.OPEN_ORDERS 
2017-12-06 21:47:21.599 UTC [DEBUG] [taskExecutor-3 ] --- [U:][M:][P:] c.c.t.s.g.MetaFunctions: evalExpr: evaluating: DELETE dbo.OPEN_ORDERS 
2017-12-06 21:47:21.599 UTC [DEBUG] [taskExecutor-3 ] --- [U:][M:][P:] c.c.t.s.g.MetaFunctions: evalExpr: ...with no parameterised variable context 
2017-12-06 21:47:21.599 UTC [DEBUG] [taskExecutor-3 ] --- [U:][M:][P:] c.c.t.s.g.MetaFunctions: uf_resolve_meta_data_item: expression result: "DELETE dbo.OPEN_ORDERS" 
2017-12-06 21:47:21.602 UTC [INFO ] [taskExecutor-3 ] --- [U:][M:][P:] c.c.t.p.l.TDMPublishLog: Execute TCORDER CLEARDOWN completed - 1 processed 
2017-12-06 21:47:21.605 UTC [INFO ] [taskExecutor-3 ] --- [U:][M:][P:] c.c.t.p.m.TDMPublishJobStart: executeActions: doing action: BeaconOrderLoad 
2017-12-06 21:47:21.605 UTC [DEBUG] [taskExecutor-3 ] --- [U:][M:][P:] c.c.t.s.g.MetaFunctions: uf_resolve_meta_data_item: evaluating expression: 
2017-12-06 21:47:21.605 UTC [DEBUG] [taskExecutor-3 ] --- [U:][M:][P:] c.c.t.s.g.MetaFunctions: evalExpr: evaluating: 
2017-12-06 21:47:21.605 UTC [DEBUG] [taskExecutor-3 ] --- [U:][M:][P:] c.c.t.s.g.MetaFunctions: evalExpr: ...with no parameterised variable context 
2017-12-06 21:47:21.605 UTC [DEBUG] [taskExecutor-3 ] --- [U:][M:][P:] c.c.t.s.g.MetaFunctions: uf_resolve_meta_data_item: expression result: "" 
2017-12-06 21:47:21.608 UTC [INFO ] [taskExecutor-3 ] --- [U:][M:][P:] c.c.t.s.p.PublishActionExecutor: Workflow Publish Action: No variables file is registered 

The following steps were followed based on the above:

  1. All the pre-publish actions associated with the generator were evaluated and they are working as expected.
  2. The TCORDER CLEARDOWN clears the space so it can be populated with the new data.
  3. BeaconOrders is not even found indicating that it has never run.

Based on this findings, prepublish execution time looks to be too high.  Need to experiment in changing from the default value of 90 seconds to something lower.

Resolution:

Based on the findings, the resolution for the results teased out of the log turned out to be the time for the filling of the data was too high.  

In general for TDM Portal publish errors, here is a quick ref check list:

1.  Check your TDMPublish.log for errors.
2.  If not in Debug mode, set debug mode (Enable Debug Logging)
3.  Rerun the issue to generate new logs.
4.  Search for errors.
5.  Evaluate all found errors to pinpoint where error is.

In this case we know where the error is.  As a result, look at the Pre-Publish actions.

StandardDefaultSetting90.JPG

Click on the 90 and change it to 30.

SuggestedFirstTestSetting30.JPG

It was re-run and it was successful.  However, this is only a method to do a one time change to test it out with.  It does not permanently save your new value. 

To Save your desired new default value, double click on the pre-publish action.

DoubleClickRowToEdit.JPG

The Edit Pre-Publish panel popups up.  Please note that an error message appears if you try to enter a blank value.  Enter a number greater than 0 seconds.

EnterNumericValueNoneBlank.JPG

 

Note that 30 was entered and Done button in the lower right corner is clicked to save changes.

ClickDoneSavesPermanentlyDefaultValueOf30.JPG

This procedure will permanently save your default values for future runs without having to change it each time.  This successfully solved this issue.