UPOPMAIL with stunnel

Forum|Forum|6 years ago
October 26, 2019
3 replies
0 views

Gianni Sandigliano
Participating Frequently

Hi Unifacers,

we need to access a mail server using crypted communication to read a mailbox.

Following suggestions found in an old post (thanks Daniel!) we setup an email proxy (stunnel) to enable UPOPMAIL to do it, because as of today UPOPMAIL does not support yet natively communication on crypted ports.

Everything went OK and we were quickly able to read a test mailbox with few messages in it!
When we tried to read a real mailbox we found a lot more messages and despite we were using ListMailX(uList), that should read mailbox in chunks of about 10MB each, this partial read DOES NOT happened: all mails headers are read in a single chunk!

Has anyone experienced this issue when using UPOPMAIL with stunnel or a similar proxy?

Thanks in advance for any suggestion or hint...

This is the code snippet we are using:

; eMail headers read...
mailNext = 1
mailLoop = 0
while mailNext
   message/nobeep "Start exec loop: %%mailLoop%%% ..."
   putmess "Start exec loop: %%mailLoop%%% ..."
   show
   mailApiHandle->ListMailX(mailHeadersMore)
   if ($status < 0)
      if ($status = -16) ; -16 indicates that data exceeds 10240000 bytes...
         if ($length(mailHeaders) > 0)
            mailHeaders = $concat(mailHeaders, "·;", mailHeadersMore) ; From 2nd chunk...
         else
            mailHeaders = mailHeadersMore ; First chunk...
         endif
      else
         message "ERROR!%%^$status=%%$status%%%%%^$procerror=%%$procerror%%%%%^$procerrorcontext=%%$procerrorcontext%%%"
         putmess "ERROR!%%^$status=%%$status%%%%%^$procerror=%%$procerror%%%%%^$procerrorcontext=%%$procerrorcontext%%%"
         return(-2)
      endif
   else
      if ($length(mailHeaders) > 0)
         mailHeaders = $concat(mailHeaders, "·;", mailHeadersMore) ; From 2nd chunk...
      else
         mailHeaders = mailHeadersMore ; First chunk...
      endif
      mailNext = 0 ; End...
   endif
   if (mailLoop >= 1) ; We are trying here to avoid to read too many messages during test...
      mailNext = 0
      break
   endif
   mailLoop = mailLoop + 1
endwhile
message/nobeep "Loop completed ..."
putmess $length(mailHeaders)
putmess mailHeaders

Regards,
Gianni

P.S. Having done further test I am almost sure a single very large chunk of headers is being transferred with ListMailX() piped through stunnel. It took about 1h (over a slow ADSL 20Mb at home) to transfer +160.000.000 bytes. This is the closing recap from stunnel.log:

2019.10.26 13:43:03 LOG5[0]: Connection closed: 368530 byte(s) sent to TLS, 160170715 byte(s) sent to socket

Gianni Sandigliano
Author
Participating Frequently
Forum|Forum|6 years ago
October 27, 2019

Hi Unifacers,

we need to access a mail server using crypted communication to read a mailbox.

Following suggestions found in an old post (thanks Daniel!) we setup an email proxy (stunnel) to enable UPOPMAIL to do it, because as of today UPOPMAIL does not support yet natively communication on crypted ports.

Everything went OK and we were quickly able to read a test mailbox with few messages in it!
When we tried to read a real mailbox we found a lot more messages and despite we were using ListMailX(uList), that should read mailbox in chunks of about 10MB each, this partial read DOES NOT happened: all mails headers are read in a single chunk!

Has anyone experienced this issue when using UPOPMAIL with stunnel or a similar proxy?

Thanks in advance for any suggestion or hint...

This is the code snippet we are using:

; eMail headers read...
mailNext = 1
mailLoop = 0
while mailNext
   message/nobeep "Start exec loop: %%mailLoop%%% ..."
   putmess "Start exec loop: %%mailLoop%%% ..."
   show
   mailApiHandle->ListMailX(mailHeadersMore)
   if ($status < 0)
      if ($status = -16) ; -16 indicates that data exceeds 10240000 bytes...
         if ($length(mailHeaders) > 0)
            mailHeaders = $concat(mailHeaders, "·;", mailHeadersMore) ; From 2nd chunk...
         else
            mailHeaders = mailHeadersMore ; First chunk...
         endif
      else
         message "ERROR!%%^$status=%%$status%%%%%^$procerror=%%$procerror%%%%%^$procerrorcontext=%%$procerrorcontext%%%"
         putmess "ERROR!%%^$status=%%$status%%%%%^$procerror=%%$procerror%%%%%^$procerrorcontext=%%$procerrorcontext%%%"
         return(-2)
      endif
   else
      if ($length(mailHeaders) > 0)
         mailHeaders = $concat(mailHeaders, "·;", mailHeadersMore) ; From 2nd chunk...
      else
         mailHeaders = mailHeadersMore ; First chunk...
      endif
      mailNext = 0 ; End...
   endif
   if (mailLoop >= 1) ; We are trying here to avoid to read too many messages during test...
      mailNext = 0
      break
   endif
   mailLoop = mailLoop + 1
endwhile
message/nobeep "Loop completed ..."
putmess $length(mailHeaders)
putmess mailHeaders

Regards,
Gianni

P.S. Having done further test I am almost sure a single very large chunk of headers is being transferred with ListMailX() piped through stunnel. It took about 1h (over a slow ADSL 20Mb at home) to transfer +160.000.000 bytes. This is the closing recap from stunnel.log:

2019.10.26 13:43:03 LOG5[0]: Connection closed: 368530 byte(s) sent to TLS, 160170715 byte(s) sent to socket

More info on the subject: retried same mailbox on a faster connection doing further search for detailed infos.

About 160MB sent from stunnel to Uniface UPOPMAIL via socket are corresponding to only about 3.5MB of mail headers recognized at application level ( $length(stringReceived) ).

Based on this new info discovered it is correct ListMailX() is not reading a second chunk of headers because it is not reaching its 10MB internal header limit.

However IMHO this solution using stunnel is really TOO slow and 10MB are really too much...
To improve user perception I would like to be able to lower the 10MB limit to 1Mb or less to let user choose to continue or to clean up the mailbox first.

Is it somehow possible to lower the 10MB limit to a smaller value?

Regards,
Gianni

PillarOfWisdom1: Inject yourself with some fantasy to NOT die of realityPillarOfWisdom2: The secret for a long healthy life is: eat half, walk double, laugh three times and love without measure! (Chinese saying)PillarOfWisdom3: Barcollo ma non mollo!

Like

+3

Ingo Stiller
Participating Frequently
Forum|Forum|6 years ago
October 28, 2019

Hi Unifacers,

we need to access a mail server using crypted communication to read a mailbox.

Following suggestions found in an old post (thanks Daniel!) we setup an email proxy (stunnel) to enable UPOPMAIL to do it, because as of today UPOPMAIL does not support yet natively communication on crypted ports.

Everything went OK and we were quickly able to read a test mailbox with few messages in it!
When we tried to read a real mailbox we found a lot more messages and despite we were using ListMailX(uList), that should read mailbox in chunks of about 10MB each, this partial read DOES NOT happened: all mails headers are read in a single chunk!

Has anyone experienced this issue when using UPOPMAIL with stunnel or a similar proxy?

Thanks in advance for any suggestion or hint...

This is the code snippet we are using:

; eMail headers read...
mailNext = 1
mailLoop = 0
while mailNext
   message/nobeep "Start exec loop: %%mailLoop%%% ..."
   putmess "Start exec loop: %%mailLoop%%% ..."
   show
   mailApiHandle->ListMailX(mailHeadersMore)
   if ($status < 0)
      if ($status = -16) ; -16 indicates that data exceeds 10240000 bytes...
         if ($length(mailHeaders) > 0)
            mailHeaders = $concat(mailHeaders, "·;", mailHeadersMore) ; From 2nd chunk...
         else
            mailHeaders = mailHeadersMore ; First chunk...
         endif
      else
         message "ERROR!%%^$status=%%$status%%%%%^$procerror=%%$procerror%%%%%^$procerrorcontext=%%$procerrorcontext%%%"
         putmess "ERROR!%%^$status=%%$status%%%%%^$procerror=%%$procerror%%%%%^$procerrorcontext=%%$procerrorcontext%%%"
         return(-2)
      endif
   else
      if ($length(mailHeaders) > 0)
         mailHeaders = $concat(mailHeaders, "·;", mailHeadersMore) ; From 2nd chunk...
      else
         mailHeaders = mailHeadersMore ; First chunk...
      endif
      mailNext = 0 ; End...
   endif
   if (mailLoop >= 1) ; We are trying here to avoid to read too many messages during test...
      mailNext = 0
      break
   endif
   mailLoop = mailLoop + 1
endwhile
message/nobeep "Loop completed ..."
putmess $length(mailHeaders)
putmess mailHeaders

Regards,
Gianni

P.S. Having done further test I am almost sure a single very large chunk of headers is being transferred with ListMailX() piped through stunnel. It took about 1h (over a slow ADSL 20Mb at home) to transfer +160.000.000 bytes. This is the closing recap from stunnel.log:

2019.10.26 13:43:03 LOG5[0]: Connection closed: 368530 byte(s) sent to TLS, 160170715 byte(s) sent to socket

BTW

 if (mailHeaders)

Is this working in all circumstances?
If mailHeader is "0", UnifAce switch to the ELSE branch

IF(<string>) is not the same then IF($length(<string>)>0) or IF(isnotnull(<string>))
UnifAce is not C 🙂

Ingo

Like

Gianni Sandigliano
Author
Participating Frequently
Forum|Forum|6 years ago
October 28, 2019

Hi Unifacers,

we need to access a mail server using crypted communication to read a mailbox.

Following suggestions found in an old post (thanks Daniel!) we setup an email proxy (stunnel) to enable UPOPMAIL to do it, because as of today UPOPMAIL does not support yet natively communication on crypted ports.

Everything went OK and we were quickly able to read a test mailbox with few messages in it!
When we tried to read a real mailbox we found a lot more messages and despite we were using ListMailX(uList), that should read mailbox in chunks of about 10MB each, this partial read DOES NOT happened: all mails headers are read in a single chunk!

Has anyone experienced this issue when using UPOPMAIL with stunnel or a similar proxy?

Thanks in advance for any suggestion or hint...

This is the code snippet we are using:

; eMail headers read...
mailNext = 1
mailLoop = 0
while mailNext
   message/nobeep "Start exec loop: %%mailLoop%%% ..."
   putmess "Start exec loop: %%mailLoop%%% ..."
   show
   mailApiHandle->ListMailX(mailHeadersMore)
   if ($status < 0)
      if ($status = -16) ; -16 indicates that data exceeds 10240000 bytes...
         if ($length(mailHeaders) > 0)
            mailHeaders = $concat(mailHeaders, "·;", mailHeadersMore) ; From 2nd chunk...
         else
            mailHeaders = mailHeadersMore ; First chunk...
         endif
      else
         message "ERROR!%%^$status=%%$status%%%%%^$procerror=%%$procerror%%%%%^$procerrorcontext=%%$procerrorcontext%%%"
         putmess "ERROR!%%^$status=%%$status%%%%%^$procerror=%%$procerror%%%%%^$procerrorcontext=%%$procerrorcontext%%%"
         return(-2)
      endif
   else
      if ($length(mailHeaders) > 0)
         mailHeaders = $concat(mailHeaders, "·;", mailHeadersMore) ; From 2nd chunk...
      else
         mailHeaders = mailHeadersMore ; First chunk...
      endif
      mailNext = 0 ; End...
   endif
   if (mailLoop >= 1) ; We are trying here to avoid to read too many messages during test...
      mailNext = 0
      break
   endif
   mailLoop = mailLoop + 1
endwhile
message/nobeep "Loop completed ..."
putmess $length(mailHeaders)
putmess mailHeaders

Regards,
Gianni

P.S. Having done further test I am almost sure a single very large chunk of headers is being transferred with ListMailX() piped through stunnel. It took about 1h (over a slow ADSL 20Mb at home) to transfer +160.000.000 bytes. This is the closing recap from stunnel.log:

2019.10.26 13:43:03 LOG5[0]: Connection closed: 368530 byte(s) sent to TLS, 160170715 byte(s) sent to socket

Hi Ingo,

Correct! ...but it worked... 🙂

Gianni

PillarOfWisdom1: Inject yourself with some fantasy to NOT die of realityPillarOfWisdom2: The secret for a long healthy life is: eat half, walk double, laugh three times and love without measure! (Chinese saying)PillarOfWisdom3: Barcollo ma non mollo!

Like

Recent badge winners

Sign up

Please log in or register:

Welcome to the Rocket Forum!

Please log in or register:

Scanning file for viruses.

This file cannot be downloaded