ES JCL SEP/Region now considered as stopped & MFCC request timed out

Forum|Forum|5 years ago
September 3, 2020
3 replies
1 view

A

Anonymous

Hello Everyone,

Sometimes the jobs are failing with the below error in scheduling end. Can you please review the below job logs from scheduling tool (Control M) and assist us.

Error log 1: RC=0209 (MFCC request timed out)

D:\\MFES\\Scripts\\MFSUBMIT.bat PROD NDB0556D
7/6/2020 12:52:35 AM MccRecvAutoAlloc: Error receiving data from server: (15) timeout expired
JCLSI0001I MFBSI Version 4.0.03 Copyright (C) 2013-2017 Micro Focus. All rights reserved. TIP_20190315
JCLSI0041I Running environment: (at 2020/07/06 00:52:04) - [EMX v4.0.01 CAScmd] - MFBSI_DIR: D:\\MFES\\PROD\\MFBSI
JCLSI0043I   Original JCL member: "d:\\MFES\\PROD\\Data\\PDS\\NM.PROD.JCLLIB\\NDB0556D.jcl"
JCLSI0044I   Substituted JCL in : "D:\\MFES\\PROD\\MFBSI\\jcltemp\\NDB0556D.jcl.000000019872"
JCLSI0003I   Submit: CASSUB.exe -stcp:mfes-pre.nmgcloud.io:5001 -j"D:\\MFES\\PROD\\MFBSI\\jcltemp\\NDB0556D.jcl.000000019872" -v2 -uOPCP\\
ROD -p*******
JCLSI0001I MFCC error in MccMfbinpCall: \\MFBSI\\jcltemp\\NDB0556D.jcl.000000019872"
JCLSI0001I MFCC request timed out
JCLSI0001I Failure submitting: "D:\\MFES\\PROD\\MFBSI\\jcltemp\\NDB0556D.jcl.000000019872"

JCLSI0077I JOB Ended with OS RC=0209. 00:52:46
JCLSI0001I === MFBSIJCL Ended. (Total Elapsed Time= 41,68 Sec,cc.)

Error log 2: RC=0228 (ES JCL SEP/Region now considered as stopped)

D:\\MFES\\Scripts\\MFSUBMIT.bat PROD NRE0002D
JCLSI0001I MFBSI Version 4.0.03 Copyright (C) 2013-2017 Micro Focus. All rights reserved. TIP_20190315
JCLSI0041I Running environment: (at 2020/09/01 13:00:04) - [EMX v4.0.01 CAScmd] - MFBSI_DIR: D:\\MFES\\PROD\\MFBSI
JCLSI0043I   Original JCL member: "d:\\MFES\\PROD\\Data\\PDS\\NM.PROD.JCLLIB\\NRE0002D.jcl"
JCLSI0044I   Substituted JCL in : "D:\\MFES\\PROD\\MFBSI\\jcltemp\\NRE0002D.jcl.000001439192"
JCLSI0003I   Submit: CASSUB.exe -stcp:mfes.nmgcloud.io:5001 -j"D:\\MFES\\PROD\\MFBSI\\jcltemp\\NRE0002D.jcl.000001439192" -v2 -uOPCPROD \\
-p*******
JCLSI0001I JCLCM0187I J0053508 NRE0002D JOB SUBMITTED (JOBNAME=NRE0002D,JOBNUM=0053508) 13:00:04
JCLSI0001I JCLCM0180I J0053508 NRE0002D Job ready for execution. 13:00:04
JCLSI0001I Processed "D:\\MFES\\PROD\\MFBSI\\jcltemp\\NRE0002D.jcl.000001439192"
JCLSI0048I JOB0053508 NRE0002D JOB Started.    ( Execution Region: PROD-BCH )   13:00:06
9/1/2020 1:01:36 PM MccRecvAutoAlloc: Error receiving data from server: (15) timeout expired
9/1/2020 1:02:11 PM MccRecvAutoAlloc: Error receiving data from server: (15) timeout expired
9/1/2020 1:02:46 PM MccRecvAutoAlloc: Error receiving data from server: (15) timeout expired
9/1/2020 1:03:21 PM MccRecvAutoAlloc: Error receiving data from server: (15) timeout expired
JCLSI0060E JOB0053508 NRE0002D ES JCL SEP/Region now considered as stopped.    13:03:21
JCLSI0062E JOB0053508 NRE0002D Job was in execution state.    13:03:21

JCLSI0077I JOB0053508 NRE0002D JOB Ended with OS RC=0228. 13:03:21
JCLSI0001I JOB0053508 NRE0002D === MFBSIJCL Ended. (Total Elapsed Time= 197,62 Sec,cc.)

Can you please help us to understand the possible reason for this issue and avoid such failures in future.

Thanks in advance

#EnterpriseDeveloper

D

Dominique Sacre
Rocketeer
19312 replies
Forum|Forum|5 years ago
September 3, 2020

Hello Everyone,

Sometimes the jobs are failing with the below error in scheduling end. Can you please review the below job logs from scheduling tool (Control M) and assist us.

Error log 1: RC=0209 (MFCC request timed out)

D:\\MFES\\Scripts\\MFSUBMIT.bat PROD NDB0556D
7/6/2020 12:52:35 AM MccRecvAutoAlloc: Error receiving data from server: (15) timeout expired
JCLSI0001I MFBSI Version 4.0.03 Copyright (C) 2013-2017 Micro Focus. All rights reserved. TIP_20190315
JCLSI0041I Running environment: (at 2020/07/06 00:52:04) - [EMX v4.0.01 CAScmd] - MFBSI_DIR: D:\\MFES\\PROD\\MFBSI
JCLSI0043I   Original JCL member: "d:\\MFES\\PROD\\Data\\PDS\\NM.PROD.JCLLIB\\NDB0556D.jcl"
JCLSI0044I   Substituted JCL in : "D:\\MFES\\PROD\\MFBSI\\jcltemp\\NDB0556D.jcl.000000019872"
JCLSI0003I   Submit: CASSUB.exe -stcp:mfes-pre.nmgcloud.io:5001 -j"D:\\MFES\\PROD\\MFBSI\\jcltemp\\NDB0556D.jcl.000000019872" -v2 -uOPCP\\
ROD -p*******
JCLSI0001I MFCC error in MccMfbinpCall: \\MFBSI\\jcltemp\\NDB0556D.jcl.000000019872"
JCLSI0001I MFCC request timed out
JCLSI0001I Failure submitting: "D:\\MFES\\PROD\\MFBSI\\jcltemp\\NDB0556D.jcl.000000019872"

JCLSI0077I JOB Ended with OS RC=0209. 00:52:46
JCLSI0001I === MFBSIJCL Ended. (Total Elapsed Time= 41,68 Sec,cc.)

Error log 2: RC=0228 (ES JCL SEP/Region now considered as stopped)

D:\\MFES\\Scripts\\MFSUBMIT.bat PROD NRE0002D
JCLSI0001I MFBSI Version 4.0.03 Copyright (C) 2013-2017 Micro Focus. All rights reserved. TIP_20190315
JCLSI0041I Running environment: (at 2020/09/01 13:00:04) - [EMX v4.0.01 CAScmd] - MFBSI_DIR: D:\\MFES\\PROD\\MFBSI
JCLSI0043I   Original JCL member: "d:\\MFES\\PROD\\Data\\PDS\\NM.PROD.JCLLIB\\NRE0002D.jcl"
JCLSI0044I   Substituted JCL in : "D:\\MFES\\PROD\\MFBSI\\jcltemp\\NRE0002D.jcl.000001439192"
JCLSI0003I   Submit: CASSUB.exe -stcp:mfes.nmgcloud.io:5001 -j"D:\\MFES\\PROD\\MFBSI\\jcltemp\\NRE0002D.jcl.000001439192" -v2 -uOPCPROD \\
-p*******
JCLSI0001I JCLCM0187I J0053508 NRE0002D JOB SUBMITTED (JOBNAME=NRE0002D,JOBNUM=0053508) 13:00:04
JCLSI0001I JCLCM0180I J0053508 NRE0002D Job ready for execution. 13:00:04
JCLSI0001I Processed "D:\\MFES\\PROD\\MFBSI\\jcltemp\\NRE0002D.jcl.000001439192"
JCLSI0048I JOB0053508 NRE0002D JOB Started.    ( Execution Region: PROD-BCH )   13:00:06
9/1/2020 1:01:36 PM MccRecvAutoAlloc: Error receiving data from server: (15) timeout expired
9/1/2020 1:02:11 PM MccRecvAutoAlloc: Error receiving data from server: (15) timeout expired
9/1/2020 1:02:46 PM MccRecvAutoAlloc: Error receiving data from server: (15) timeout expired
9/1/2020 1:03:21 PM MccRecvAutoAlloc: Error receiving data from server: (15) timeout expired
JCLSI0060E JOB0053508 NRE0002D ES JCL SEP/Region now considered as stopped.    13:03:21
JCLSI0062E JOB0053508 NRE0002D Job was in execution state.    13:03:21

JCLSI0077I JOB0053508 NRE0002D JOB Ended with OS RC=0228. 13:03:21
JCLSI0001I JOB0053508 NRE0002D === MFBSIJCL Ended. (Total Elapsed Time= 197,62 Sec,cc.)

Can you please help us to understand the possible reason for this issue and avoid such failures in future.

Thanks in advance

#EnterpriseDeveloper

Unfortunately there is not enough detail here to allow for proper analysis. From what can be seen, both jobs ended up with some type of network communications error when trying to either submit or monitor the submitted job within Enterprise Server. It would probably make sense to open a tick with support on the next occurrence. When you do, please run the utility MFESDIAGS which will gather all of the pertinent logs and files and zip them up so you can attach them with the incident. At a minimum CONSOLE.LOG and LOG.HTML are needed, plus any MFBSI job logs. Hopefully that will provide the details needed to see what may be causing this or at least point to the next set of diagnostics to gather.

MFESDIAGS can be downloaded from https://supportline.microfocus.com/examplesandutilities/TroubleShootingpack/MFESDIAGS.zip

You may also want to reach out to you sales rep to discuss the possibility of some consulting help to get the environment more stable. They could probably assist with the CURL question you had as well

A

Anonymous
0 replies
Forum|Forum|5 years ago
September 3, 2020

Unfortunately there is not enough detail here to allow for proper analysis. From what can be seen, both jobs ended up with some type of network communications error when trying to either submit or monitor the submitted job within Enterprise Server. It would probably make sense to open a tick with support on the next occurrence. When you do, please run the utility MFESDIAGS which will gather all of the pertinent logs and files and zip them up so you can attach them with the incident. At a minimum CONSOLE.LOG and LOG.HTML are needed, plus any MFBSI job logs. Hopefully that will provide the details needed to see what may be causing this or at least point to the next set of diagnostics to gather.

MFESDIAGS can be downloaded from https://supportline.microfocus.com/examplesandutilities/TroubleShootingpack/MFESDIAGS.zip

You may also want to reach out to you sales rep to discuss the possibility of some consulting help to get the environment more stable. They could probably assist with the CURL question you had as well

You are correct Steve

I am observing below communication/network related entries (multiple times) in log -1 files whenever I am seeing job failures with timeout

•2020-09-01 13:04:16.938 [1] CCI error 8 in send on conversation 37074 for listener Web Services and J2EE(5001):
CCITCP-0015E A CCITCP transport failure has occurred. (CCI 15, Ext 10053)
•2020-09-01 13:04:16.938 [1] Conversation 37074 closed by client at 172.21.153.131:49424 or failed due to broken network connection
•2020-09-01 13:04:16.938 [1] Internal error BAD_STATE (9) in SendResponse

•2020-09-01 13:43:26.226 [1] CCI error 8 in send on conversation 37568 for listener Web Services and J2EE(5001):
CCITCP-0015E A CCITCP transport failure has occurred. (CCI 15, Ext 10054)
•2020-09-01 13:43:26.226 [1] Conversation 37568 closed by client at 172.21.153.131:50227 or failed due to broken network connection
•2020-09-01 13:43:26.226 [1] GkProcessConversation: Error sending or receiving data for conversation 37568 for listener "Web Services and J2EE": COMMS: communications component error

Additional Observations:

1. Sometimes noticed Empty SEP executing with busy state in ESMAC table

2. Job failure occurs only at client end (control M)

We are not exactly sure what causing this communication issue between server and client. Kindly provide your thoughts based on the above log entry from log-1 file

A

Anonymous
0 replies
Forum|Forum|5 years ago
September 29, 2020

You are correct Steve

I am observing below communication/network related entries (multiple times) in log -1 files whenever I am seeing job failures with timeout

•2020-09-01 13:04:16.938 [1] CCI error 8 in send on conversation 37074 for listener Web Services and J2EE(5001):
CCITCP-0015E A CCITCP transport failure has occurred. (CCI 15, Ext 10053)
•2020-09-01 13:04:16.938 [1] Conversation 37074 closed by client at 172.21.153.131:49424 or failed due to broken network connection
•2020-09-01 13:04:16.938 [1] Internal error BAD_STATE (9) in SendResponse

•2020-09-01 13:43:26.226 [1] CCI error 8 in send on conversation 37568 for listener Web Services and J2EE(5001):
CCITCP-0015E A CCITCP transport failure has occurred. (CCI 15, Ext 10054)
•2020-09-01 13:43:26.226 [1] Conversation 37568 closed by client at 172.21.153.131:50227 or failed due to broken network connection
•2020-09-01 13:43:26.226 [1] GkProcessConversation: Error sending or receiving data for conversation 37568 for listener "Web Services and J2EE": COMMS: communications component error

Additional Observations:

1. Sometimes noticed Empty SEP executing with busy state in ESMAC table

2. Job failure occurs only at client end (control M)

We are not exactly sure what causing this communication issue between server and client. Kindly provide your thoughts based on the above log entry from log-1 file

Hi Steve,

Greetings!!!

As per micro focus support, added MFC_CONNECT_TIMEOUT and MFC_TIMEOUT environment variables in ES config & MFBSI script to resolve this timeout related issue.

Also increased the time value (In sec) as mentioned below

MFC_CONNECT_TIMEOUT = 300

MFC_TIMEOUT = 300

Monitored the system since 9/15 (change implementation date) and we are not observing job failures with MFCC timeout error.

Thanks much for your support as always

Sign up

Please log in or register:

Welcome to the Rocket Forum!

Please log in or register:

Scanning file for viruses.

This file cannot be downloaded