Friday 13 October 2017

How to Read a Flowfile Line by Line in NiFi

Edit: This original code posted was not accurate. IOUtils.toString(in) returns the whole file, therefore it must be split by \r\n or \n (EOL) into an array and this can be added to the lines arraylist using a foreach loop as shown below:

@Overridepublic void onTrigger(final ProcessContext context, final ProcessSession session) throws ProcessException {
    FlowFile flowFile = session.get();
    if ( flowFile == null ) {
        return;
    }

    ArrayList<String> lines = new ArrayList<>();
    session.read(flowFile, new InputStreamCallback() {
        @Override        public void process(InputStream in) throws IOException {
            try{
                String text = IOUtils.toString(in);
                String _lines[] = text.split("\\r?\\n");
                for (String l : _lines) {
                    lines.add(l);
                }
            }catch(Exception ex){
            }
        }
    });

Alternative:

ArrayList<String> lines = new ArrayList<>();
session.read(flowFile, new InputStreamCallback() {
    @Override    public void process(InputStream in) throws IOException {
        try{
            String text = IOUtils.toString(in);
            String _lines[] = text.split("\\r?\\n");
            lines.addAll(Arrays.asList(_lines));
        }catch(Exception ex){
        }
    }
});

No comments:

Post a Comment