Problem with sending batches of email

Hi,
I have an application that is rendering HTML reports supposed to be send over via mail to my internal subscriber users.
The HTML rendering works well ( output tested on a browser is what it has to be ).
When I do send the contents of the rendered report as message in an email I have strange behaviour where it looks the contents is recursively parsed and reprinted by the smtp server ( since the original generated file on my server is perfectly correct ), on this one, sending batches of mails is working as expected ( no timeout issues as well btw, works on development AND production ), except the contents is not good due to above. This may be me not being an expert on how to handle “evil” HTML emails, so I’ve tried a different approach, by sending the report as attached file to my users.

The problem I have :
on development environment, which is Windows, it does not work well with file attachments, only the first mail is sent out by the system. On live environment, ubuntu 16.04, PHP 7.1 ( slight difference in PHP version compared to dev environment, which is a 7.0.xxx ), only the first email is sent out, all other files are said sent of by {sc_mail_ok} but they are actually not sent.

my error_log is clean.
my apache server on production has SSL (self signed) certificate.

What I’ve done :
Assuming Swift is quite a good mail sending solution and considering it almost works for me, I did not try to integrate PHPMailer or anything else such.

I’ve been investigating the Swift components and the scriptcase generated index.php that issues the mails to try and see if there were not missing class destructors, or any garbage collectors to be called, modifying directly the index.php at the end of the mail send procedure I have and pushing it to the server :

$Con_Mail->reset();
$Con_Mail->stop();
$Send_Mail = null;
$Con_Mail = null;
$Mens_Mail = null;

calling garbage collector, it did not work.

After different tries, I confirmed the batching issue came from files attachment on my server and dev.
I have seen that the memory used was increasing after each call to sc_mail_send, so I have tried to investigate how to release memory ( although I have no low memory issue declared ), considering the destroy activities were not ok.

So, I have tried to clean the attachment objects by slightly modifying $Mens_Mail->attach(Swift_Attachment::fromPath($NM_dest)); in the scriptcase generated code, so to be able to detach, then unset or null these attachnemnts at the end of the same procedure.
It did nothing more either ! only first file being sent, and then nothing.

From another post on this perspective, I’ve also tried to disable my ssl site on apache2 to see whether SSL certificates conflicts could have caused this. I’ve had similar behaviour.

What I’m now looking into

Since it’s a difference on different configurations, I’m now reviewing the configurations between prod and dev.
I’m however not clear whether ApacheScriptcase9 behaves exactly as standard Apache 2 [HR][/HR]
my code to generate reports and call my send library

{loop start}
$mail_message ="Hi,<br>This is an automated email sent ";
$mail_message.="<br><br>Thanks<br>System.";

// generate document

$folder = $_SERVER['DOCUMENT_ROOT'];
$folder.= $_SESSION['scriptcase']['test_email_system']['glo_nm_path_imag_temp']."/";
$file = $folder.session_id()."_".$count.".html";

// Generate report and put it to file
$Msg=renderHtmlReport($HTMLReport,$initVar,$initVa l);
file_put_contents($file,$Msg);
$count++;

// test visually generated HTML report
echo $Msg;

// Check and send
echo " Sending to ".$EmailAddress." [".$Title."] ".$mail_message."<br>";
echo " file : ".$file."<br>";
$ret = sendMail($EmailAddress,$EmailAddress,$Title,$mail_ message,$file);
echo " returns ".$ret."<br>";
unlink ($file);
{loop_end}

[HR][/HR]
My code for the sendMail function that I made in a public library

 function sendMail($toMail,$toName,$title,$Message,$file)
{


// --------------------------SMTP Server parameters ------------------------
$SQL = "SELECT * FROM ADM_SMTP_SERVER WHERE IS_SELECTED=1 LIMIT 1";
sc_lookup(res,$SQL);

if(empty($res)) return;
$Server = $res[0][1];
$User = $res[0][2];
$Pass = sc_decode($res[0][3]);
$SystemMail = $res[0][4];

$mail_smtp_server = $Server; // SMTP server name or IP address
$mail_smtp_user = $User; // SMTP user name
$mail_smtp_pass = $Pass; // SMTP password
$mail_from = $SystemMail; // From email
$mail_to = $toMail; // To email
$mail_subject = $title; // Message body
$mail_format = 'H'; // Format: (T)ext or (H)tml
$mail_message = $Message."<hr>";
$attachments = $file;

// "desparate attempt" to clear pointers beforehand
$Send_Mail = null;
$Con_Mail = null;
$Mens_Mail = null;

sc_mail_send( $mail_smtp_server,
$mail_smtp_user,
$mail_smtp_pass,
$mail_from,
$mail_to,
$mail_subject,
$mail_message,
$mail_format,
'','',
'465',
'S',
$attachments);

// "desparate attempt" to clear pointers afterhand
$Con_Mail->stop();
$Send_Mail = null;
$Con_Mail = null;
$Mens_Mail = null;

// Collects errors on sending
if ({sc_mail_ok}) {
$ret=0;
} else {
$ret=1;
}

return($ret);
} 

Details on previous investigations

A small complement on the changes initally done to memory management with Swift in my index.php file :
Keep track of attachment files done ( direct edit in the code ). This is unfortunately lost each time application is generated by Scriptcase…

Instead of

foreach ($Temp_mail as $NM_dest)
{
if (!empty($NM_dest))
{
$Mens_Mail->attach(Swift_Attachment::fromPath($NM_dest));
}
}

I have written :

$n=0; $swiftAttach = array();
foreach ($Temp_mail as $NM_dest)
{
if (!empty($NM_dest))
{
$swiftAttach[$n] = Swift_Attachment::fromPath($NM_dest);
$Mens_Mail->attach($swiftAttach[$n]);
$n++;
}
}

And then at the end of the sending procedure source code, I did :
for ($j=0;$j<$n;$j++) {
$Mens_Mail->detach($swiftAttach[$j]);
$swiftAttach[$j]->__destruct();
$swiftAttach[$j]=null;
}
$Con_Mail->stop();
$Con_Mail->__destruct();
$Con_Mail = null;
$Mens_Mail->__destruct();
$Mens_Mail = null;

Calling to memory_get_usage(false) before and after call shows a difference with and without using detach() and __destruct(), but it seems not comprehensive.

I cannot destruct properly the source object $Send_Mail so to be clear it’s not the right direction… Even if I strongly suspect it has something to do with sending attachments. [HR][/HR]
apache SSL site deactivation was done using # sudo a2dissite mysslconfsite and # sudo systemctl restart apache2 [HR][/HR]
Furthermore a clue, finally : Swift does not like my HTML attachment files :frowning:

After all these digarounds memory, and back to basics, I decided to send small test files to test instead of the generated reports ( all my generated report files are above 4kB, way less than 100kB anyway in HTML format ). And this has been working fully !! as a next trial, I’ve tested sending the report files encoded in base 64 and it has worked, all mails being sent out properly. Not a problem of memory finally ( probably an old reflex from C++ development I’ve had - at least I know now better how Swift works :slight_smile: ).

And the solution was simple

Problem I have with Swift sending multiples mails in a batch is therefore related to my HTML file format … and it was something quite “stupid”.
After I modified the generation code to add the HTML headers and encoding, it was all sorted, at least !! Because my generation procedure did not encapsulate the generated contents within standard HTML tags. When I think I was looking at memory and complex issues… anyway, I’ve got it now. And I suspect this could help some of you at some point.

$Header = "<!DOCTYPE html><HTML><BODY>";
$Header.= "<meta http-equiv="Content-Type" content="text/html;charset=UTF-8">";
$Footer = "</BODY></HTML>";
$Message = $Header.($Msg).$Footer;

file_put_contents($file,$Message);

[LEFT]
[SIZE=13px][B]​So, I did not investigate further about the configuration files !!

Notes :[/B]
​I also tested why my HTML contents, properly rendered when sent as a file ( from now on ) sent as simple mail body was screwed. Unfortunately it still behaves oddly with all the required HTML tags. I think it’s related to how SMTP servers process the message contents upon reception. Did not look any further since I’m happy with the solution I have. ​In any case if someone has a hint about this, feel free to add to my post.

Cheers[/SIZE][/LEFT]