I’m trying to make some assertions over my application log messages. That’s how I found it out. So, yes, the pipeline is intentionally failing.

Curiously, I didn’t have this issue on previous crunch versions (0.11.0-cdh5.4.2). I just think that the final error message may be a little bit misleading to someone that does not know Crunch and the application.

Last but not least, as someone has explicitly said that dotfile is built even if its generation is disabled, I would like to understand why.

Thank you!

(I not so available right now, but I may report/fix it as soon as I have some free time).

Ingénieur Développement
Volt Europe pour Médiamétrie

De : Gabriel Reid [mailto:[EMAIL PROTECTED]]
Envoyé : mercredi 21 juin 2017 15:05
Objet : Re: Dotfile default generation

Hi Giovanni,

I believe that the building of the general job dotfile was added to Crunch before the default disabling of the other dotfiles was added, and so it is always created to maintain backwards compatibility.

Please feel free to log a ticket (or even better, provide a patch) to have have this changed at https://issues.apache.org/jira/browse/CRUNCH.

By the way, if exceptions like the one you mentioned ("Problem creating debug dotfile: Input source Avro(…) does not exist!”) are being logged, doesn't this mean that your pipeline execution is also failing?

- Gabriel
2017-06-21 14:41 GMT+02:00 Giovanni GATTI PINHEIRO <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>>:

I would like to understand the reason why DotfileUtil.buildPlanDotfile builds the dotfile despite of dotfile-debug mode is enabled or not.

This may generate weird log messages such as

“Problem creating debug dotfile: Input source Avro(…) does not exist!”

if input file is missing.

Thank you in advance.

Ingénieur Développement
Volt Europe pour Médiamétrie

Buropolis Les Bouillides - 1240, Route des Dolines
06560 Valbonne Sophia-Antipolis - France


[cid:[EMAIL PROTECTED]7340]<https://mediametrie.letsignit.com/r/5/c2b7ffb6-8501-42ca-9278-776bb854246b/ed470b86-9a08-46a8-b51f-5e98f4a9daf7>

Nous vous rappelons que les résultats de Médiamétrie sont et demeurent sa propriété : ils sont protégés au double
titre du droit d'auteur et de la protection des bases de données. Ce message est confidentiel et établi à
l'intention de ses destinataires. Tout message électronique étant susceptible d'altération, la société Médiamétrie
décline toute responsabilité s'il a été altéré, déformé ou falsifié.

We remind you that the results produced by Médiamétrie are and remain its sole property covered by both copyright
and databases protection. This message is confidential and intended solely for the adressees. E-mails are susceptible
to alteration. Neither Médiamétrie company shall be liable for the message if altered, changed or falsified.
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB