Goto devoxx

@MSkarsaune#GotoJava

goto java;- Java compilation under the hood

Martin Skarsaune@kantega.no

Kantega – Norwegian software craftsmanship since 2003

高馬丁

GOTO Statements in Java!

GOTO Statement –Objective

• Syntax goto identifier;• Runtime• Program control moves to target statement

• Other Semantics • Target (statement label) must be defined in

same scope, compilation error if not• Potential circular gotos should give

compilation warning.

GOTO Statement –Means

• OpenJDK• Open source Java implementation• Javac is implemented in plain Java

• Modify compiler to support GOTO

public class GotoSuccess {

public static void main(String[] args) {one: System.out.print("goto ");two: System.out.print(”DEV");goto four;three: System.out.print(”UK");goto five;four: System.out.print(”OXX ");goto three;five: System.out.print("!");

Success Case

GOTOAHEAD

Dijkstra 1968:

What is a Compiler?

Compiler

What is a Compiler?

frontend

backend

Parse Enter Process Attribute Flow Desuga

rGenerat

processAnnotations(enterTrees(… parseFiles(…))),…)

…generate(desugar(flow(attribute(…))))

rGenerat

• Syntax: goto identifier;• First convert character stream to token

stream (Scanner.java)

GOTO IDENTIFIER SEMI

[g][o][t][o][ ][f][o][u][r][;]

rGenerat

• goto is already a reserved word in Java!• Lucky for us, goto is therefore defined as a

TokenKind. • Tokens.java:141: GOTO("goto")• The scanner therefore works out of the box!

final Scanner scanner=factory.newScanner(”goto identifier;”, false);

rGenerat

...import com.sun.tools.javac.parser.Scanner;...

scanner.nextToken();

rGenerat

scanner.nextToken();assertThat(”First token is GOTO”, scanner.token().kind, is(GOTO));

rGenerat

scanner.nextToken();assertThat("Second token is IDENTIFIER", scanner.token().kind,

is(IDENTIFIER));

rGenerat

is(IDENTIFIER));

rGenerat

is(IDENTIFIER));

scanner.nextToken(); assertThat("Third token is SEMI", scanner.token().kind, is(SEMI));

rGenerat

CompilationUnit

ClassDefinition

FieldDefintion

MethodDefinition

Signature

IfStatement

rGenerat

visitClassDef(..)

visitMethodDef(..)

visitIf(..)

Abstract Syntax Tree

[g][o][t][o][ ][f][o][u][r][;]

GOTO IDENTIFIER SEMIWikipedia: “the visitor design pattern is a way of separating an algorithm from an

object structure on which it operates”

rGenerat

ClassInterface

rGenerat

Interface based visitors

rGenerat

Class based visitors

public void visitGoto(JCGoto tree) { try {

print("goto " + tree.label + ";"); } catch (IOException e) {

throw new UncheckedIOException(e); }

public void visitGoto(JCGoto tree) { //TODO implement

rGenerat

public static class JCLabeledStatement extends JCStatement implements LabeledStatementTree {…public GotoResolver handler;…}

public class GotoResolver { Map<GotoTree, Name> gotos; Map<Name, LabeledStatementTree> targets; List<StatementTree> statementsInSequence; ...}

Helper object

rGenerat

JavacParser.parseStatement()

case GOTO: { nextToken(); Name label = ident(); JCGoto t = to(F.at(pos).Goto(label, getGotoResolver())); accept(SEMI); return t; }

TreeMaker.java:

public JCGoto Goto(Name label, GotoResolver resolver) { JCGoto tree = new JCGoto(label, resolver); tree.pos = pos; return tree;}

@YourTwitterHandle@MSkarsaune#GotoJava

rGenerat

• Basic sanity testing of compilation unit

• File name and folder location

• Duplicate class names• Corrections• Add default constructor if

no constructors are declared

rGenerat

Default Constructorpublic class SimpleClass {

rGenerat

Default Constructorpublic class SimpleClass { public SimpleClass() { super(); }}

rGenerat

• Process Annitations• Annotation processing API• Part of ordinary javac process since Java 1.6

• Plugin API (see javac documentation)

rGenerat

• Output controlled by command line switches

- proc:only – only process annotations, do not compile

- proc:none – do not process annotations

rGenerat

Annotation processor Example• ensure that all fields with @Config

annotation are primitive or strings

@Configprivate String sender;//ok @Configprivate int retries;//ok @Configprivate String[] receivers;//not ok

rGenerat

META-INF/services/javax.annotation.processing.Processor:

org.kantega.reststop.apt.ConfigFieldProcessor

rGenerat

ConfigFieldProcessor.java :

@SupportedAnnotationTypes("org.kantega.reststop.api.Config")@SupportedSourceVersion(SourceVersion.RELEASE_7)public class ConfigFieldProcessor extends AbstractProcessor { //... public boolean process(Set<? extends TypeElement> annotations, RoundEnvironment roundEnv) {

//... if(! isPrimitive(type) && ! isString(type) ) {

processingEnv.getMessager().printMessage(Diagnostic.Kind.ERROR, "@Config annotated field must primitive, boxed or String”, element)

rGenerat

Simply add to compilation path:

<dependency> <groupId>org.kantega.reststop</groupId> <artifactId>reststop-annotation-processor</artifactId> <version>${reststop.version}</version> <scope>compile</scope></dependency>

rGenerat

• Devoxx BE 2014 : “Plugging into the Java Compiler” , •https://www.parleys.com/tutorial/plugging-java-compiler-1

•Google the session title for additional resources.

rGenerat

• Attribution• Semantic checks

–Types–References

• Corrections–Add required calls to super constructor

rGenerat

Ensure target label exists in current scopepublic class GotoMissingLabel {

public static void main(String[] args) {one: System.out.print("goto ");two: System.out.print(”DEV");goto six;three: System.out.print(”UK");goto five;four: System.out.print(”OXX ");goto three;five: System.out.print("!");

rGenerat

Attr.java: @Override public void visitGoto(JCGoto that) {

that.findTarget();if(that.target==null)

log.error(that.pos(), "undef.label", that.label);result = null;

class JCGoto: …public void findTarget() { this.target = (JCLabeledStatement)this.handler.findTarget(this);}

rGenerat

Flow analysis• Detect unreachable code• Verify assignments• Ensure proper method return• Verify (effectively) final• Check exception flow

rGenerat

Detect circular gotospublic class GotoCircularWarning {

public static void main(String[] args) { one: System.out.print("goto "); two: System.out.print(”DEV"); goto four; three: System.out.print(”UK"); goto five; four: System.out.print(”OXX "); goto three; five: System.out.println("!"); goto one;//forms infinite loop }}

rGenerat

Flow.FlowAnalyzer class:

compiler.properties:

@Override public void visitGoto(JCGoto tree) { if (tree.handler.detectCircularGotoPosition(tree))

log.warning(tree.pos, "circular.goto"); }

...compiler.warn.circular.goto=circular goto...

rGenerat

TransTypes Unlambda LowerErase generic types

public class Bridge implements Comparator {

public interface Comparator<T> {tint compare(T o1, T o2);

}or<T> {

rGenerat

TransTypes Unlambda LowerErase generic types

public class Bridge implements Comparator<Integer> {

public int compare(Integer first, Integer second) {return first - second;

public interface Comparator<T> {tint compare(T o1, T o2);

}or<T> {

rGenerat

TransTypes Unlambda LowerErasure - Runtime

rGenerat

TransTypes Unlambda LowerErasure - Bridge

} /*synthetic*/ public int compare(Object first, Object second) { return this.compare((Integer)first, (Integer)second); }

rGenerat

TransTypes Unlambda LowerExtract inner class

public class Outer {private void foo() { }

public Runnable fooRunner() {return new Runnable() {

public void run() {foo();

rGenerat

public Runnable fooRunner() {return new Runnable() {

public void run() {foo();

rGenerat

public Runnable fooRunner() {return new Outer$1(this);

class Outer$1 implements Runnable {final Outer this$0;

Outer$1(final Outer this$0) { this.this$0 = this$0; super(); } public void run() {

this$0.foo(); }}

rGenerat

this$0.foo(); }}

rGenerat

}/*synthetic*/static void access$000(

Outer x0) { x0.foo(); }

Outer.access$000(this$0); }}

rGenerat

TransTypes Unlambda LowerBoxing

List<Integer> list = Arrays.asList(1, 2);

for (Integer i : list) {System.out.println("Double: " + i * 2);

rGenerat

List<Integer> list = Arrays.asList(1, 2);

public static <T> List<T> asList(T... a) { return new ArrayList<>(a); }

rGenerat

List<Integer> list = Arrays.asList(Integer.valueOf(1), Integer.valueOf(2));

rGenerat

TransTypes Unlambda LowerUnboxing

rGenerat

TransTypes Unlambda LowerUnboxing

for (Integer i : list) {System.out.println("Double: " + i.intValue() * 2);

rGenerat

TransTypes Unlambda LowerVarargs

public static <T> List<T> asList(T... a) { return new ArrayList<>(a); }

rGenerat

TransTypes Unlambda LowerVarargs

public static <T> List<T> asList(T... a) { return new ArrayList<>(a);}

rGenerat

TransTypes Unlambda LowerVarargs - runtime

List<Integer> list = Arrays.asList(new Integer[]{Integer.valueOf(1), Integer.valueOf(2)});

rGenerat

TransTypes Unlambda LowerFor each loop

rGenerat

public interface Iterable<T> { Iterator<T> iterator();}

rGenerat

for (;;) {Integer i System.out.println("Double: " + i.intValue() * 2);

rGenerat

for (Iterator i$ = list.iterator();;) {Integer i System.out.println("Double: " + i.intValue() * 2);

rGenerat

for (Iterator i$ = list.iterator(); i$.hasNext();) {Integer i System.out.println("Double: " + i.intValue() * 2);

rGenerat

for (Iterator i$ = list.iterator(); i$.hasNext();) {Integer i = (Integer)i$.next(); System.out.println("Double: " + i.intValue() * 2);

rGenerat

TransTypes Unlambda LowerEnums

public enum Status {YES, NO, MAYBE

rGenerat

TransTypes Unlambda LowerEnums - constructor

public enum Status {YES, NO, MAYBEprivate Status(String $enum$name, int $enum$ordinal) {

super($enum$name, $enum$ordinal); }}

public static final Status TRUE = new Status("TRUE", 0);public static final Status FALSE = new Status("FALSE", 1);public static final Status MAYBE = new Status("MAYBE", 2);

rGenerat

TransTypes Unlambda LowerEnums - valueOf

super($enum$name, $enum$ordinal); } public static Status valueOf(String name) { return (Status)Enum.valueOf(Status.class, name); }}

rGenerat

TransTypes Unlambda LowerEnums - values

super($enum$name, $enum$ordinal); } public static Status valueOf(String name) { return (Status)Enum.valueOf(Status.class, name); } private static final Status[] $VALUES = new Status[]{ Status.YES, Status.NO, Status.MAYBE};

public static Status[] values() { return (Status[])$VALUES.clone(); }}

rGenerat

TransTypes Unlambda LowerEnum switch statement

public class SwitchStatus {

void switchStatus(Status status) {

switch (status) { case MAYBE: return;

default: break; } }}

rGenerat

switch (status) { case MAYBE: return;

rGenerat

switch (SwitchStatus$1.$SwitchMap$Status[(status).ordinal()]) { case 1: return;

class SwitchStatus$1 {

rGenerat

class SwitchStatus$1 {static final int[] $SwitchMap$Status = new

int[Status.values().length];

rGenerat

int[Status.values().length]; [0][0][0]

rGenerat

int[Status.values().length]; [0][0][1]static { try { SwitchStatus$1.$SwitchMap$Status[Status.MAYBE.ordinal()] = 1; } catch (NoSuchFieldError ex) { } }}

rGenerat

• Organize initializers• String concatenation• Generate bytecodes

rGenerat

• Organize <init> (Constructor)public class InstanceInitialization { String key="key"; String value; public InstanceInitialization(String value) { this.value = value; }

rGenerat

• Organize <init> (Constructor)public class InstanceInitialization { String key="key"; String value; public InstanceInitialization(String value) {

super(); this.value = value; }

rGenerat

• Organize <init> (Constructor)public class InstanceInitialization { String key; String value; public InstanceInitialization(String value) {

super(); key = ”key”; this.value = value; }

rGenerat

• Organize <init> (Constructor)public class InstanceInitialization { String key; String value; public void <init> () {

super(); key = ”key”; this.value = value; }

rGenerat

• Organize <clinit> (static initialization)public class StaticInitialization { static String key="key"; static { init(); }}

rGenerat

• Organize <clinit> (static initialization)public class StaticInitialization { static String key; static { key="key"; init(); }}

rGenerat

• Organize <clinit> (static initialization)public class StaticInitialization { static String key; static void clinit() { key="key"; init(); }}

rGenerat

String concatenationSource code “Generated code”

”string” + value

rGenerat

”string” + value new StringBuilder()

rGenerat

”string” + value new StringBuilder() .append(”string”)

rGenerat

”string” + value new StringBuilder() .append(”string”) .append(value)

rGenerat

”string” + value new StringBuilder() .append(”string”) .append(value) .toString()

rGenerat

• Goto generation–Luckily for us there is a GOTO byte code–goto <addr>

rGenerat

if (<test>) { <ifblock>} else { <elseblock>}<codeafter>

Source code Byte codeCI

9 ifeq<elseblock>

goto<stackmap>

22 <ifblock>

29 <codeafter>

Java >= 1.6:Stack map frames must be embedded at target of jump instruction

Code generator (Code.java) marked as not alive.Goto instruction added to list of pending jumps (Chain.java)

Pending jumps processed

Normal GOTO usage

rGenerat

…label: <somecode>…goto label;

… goto 20

Used by goto?Must emit stackmap

Emit goto to label and turn code generation on again

GOTO scenario 1 : jump back

rGenerat

GOTO scenario 2 : jump forward

…goto label; …label: <somecode>

……

29 <somecode>

Label position not yet known?• emit goto• add to list of pending gotos• turn generation on again

29Label used? • emit stack frame • patch pending

rGenerat

• Goto generation–Gen.java , visitor for code generation–Modify for LabelledStatement–Add implementation for Goto

rGenerat

• Gen.java – Labelled Statement

public void visitLabelled(JCLabeledStatement tree) { // if the label is used from gotos, have to emit stack map if (tree.handler.isUsed(tree)) code.emitStackMap(); … }

rGenerat

• Gen.java – Goto

public void visitGoto(JCGoto tree) { tree.handler.addBranch(new Chain(code.emitJump(goto_), null, code.state.dup()), tree.target); //normally goto marks code as not alive, turn generation on code.entryPoint();}

Target position known?• Yes – patch immediately• No – add to list of pending

ambdaλ

rGenerat

TransTypes Unlambda Lower

• Lambda implementation in Java 8–Language change–Compilation–Runtime support

• Many interesting design considerations

rGenerat

TransTypes Unlambda LowerSimple Example

public Comparator<String> lambdaExample() { return (String a, String b) -> a.compareTo(b);}

rGenerat

TransTypes Unlambda LowerLambdaToMethod.java

public Comparator<String> lambdaExample() { return (String a, String b) -> a.compareTo(b);}

/*synthetic*/ private static int lambda$lambdaExample$0( , ) { return ;}

final String afinal String b

a.compareTo(b)

rGenerat

TransTypes Unlambda LowerRuntime

public Comparator<String> lambdaExample() { return <invokedynamic>LambdaMetafactory.metafactory(}

/*synthetic*/ private static int lambda$lambdaExample$0( final String a, final String b) { return a.compareTo(b);}

public interface Comparator {/*erased*/ int compare(Object o1, Object o2);}

Lookup(LambdaExample), /*caller*/ "compare", ()Comparator, /*MethodType*/ (Object,Object)int, /*MethodType*/ lambda$lambdaExample$0, /*MethodHandle*/ (String,String)int); /*MethodType*/

final class LambdaExample$$Lambda$1/1834188994 implements Comparator { private LambdaExample$$Lambda$1/1834188994() public int compare(Object,Object)}

rGenerat

Runtime implementation

LambdaMetaFactory• metaFactory(…)• altMetaFactory(…)

InnerClassLambdaMetafactory

Lambda Class

rGenerat

SerializationLambda Instance

Serialize

SerializedLambda

Deserialize

LambdaMeta

Factory

rGenerat

• Possible to back port ?–Capture generated class ?–Compile time generation of inner class!

rGenerat

TransTypes Unlambda LowerStep 1: Source.java

public boolean allowLambda() { return compareTo( ) >= 0;}

JDK1_8JDK1_5

rGenerat

TransTypes Unlambda LowerStep 2: Special handling

boolean mustBackportLambda() { return this.target.compareTo(Target.JDK1_8) < 0;}

rGenerat

TransTypes Unlambda LowerStep 3: Call backport

if(!this.attr.mustBackportLambda()) { result = makeMetafactoryIndyCall(...);}else{ result = new LambdaBackPorter(...).implementLambdaClass(...);}

rGenerat

TransTypes Unlambda LowerExample implementation

private static final class Lambda$$2 implements Comparator<String> {

Lambda$$2 { super(); } public int compare(String arg0, String arg1) { return LambdaExample.lambda$lambdaExample$0(arg0, arg1); } public int compare(Object o1, Object o2) { return this.compare((String)o1, (String)o2); }}

rGenerat

TransTypes Unlambda LowerExample invoking

public Comparator<String> lambdaExample() { }

return LambdaMetafactory.metafactory(...);return new Lambda$$289();

ambdaλ

@YourTwitterHandle#DVXFR14{session hashtag} @MSkarsaune#GotoJava

Wrap Up

• The Java compiler is written in Java• Compilation is done in phases• Programming language advances (syntactic

sugar) require good compiler support• Lambdas are compiled in a forward compatible

manner

Resources• OpenJDK

• Source: http://hg.openjdk.java.net/• Compiler hacking tutorial

• http://www.ahristov.com/tutorial/java-compiler.html• Lambda design discussion

• http://cr.openjdk.java.net/~briangoetz/lambda/lambda-translation.html

• Sample code:• https://github.com/skarsaune/goto

• Slide pack:• http://www.slideshare.net/MartinSkarsaune

https://wall-simple.sli.do/#/event/cmnxxfl0/section/18316/questions

for yo

http://lanyrd.com/profile/martinskarsaune/

Goto devoxx

Internet

Microservices IRL - Devoxx-fr 2016

Devoxx Booth Slide Loop

Devoxx Devops University 2012

Les Cast Codeurs Devoxx France 2017

Devoxx test ng

Devops mycode devoxx-france-2015-v2

synyx - devoxx for kids bedcon15 - devoxx for... · 2016. 1. 15. · @Devoxx4KidsDE Devoxx für Kinder Java-Konferenz /**/ i++; if(v = null) return result;

JHipster from trenches - DevoXX 2017

Asynchronismes en JavaScript - Devoxx 2015

Devoxx France 2014 - Votez pour nous !

Javaslang @ Devoxx

goto java; (Jfokus)

Devoxx France 2015 - Développement web en 2015

L'expérience Utilisateur est impotante pour nous (Devoxx)

Devoxx France - Nouvelles du Front

Keynote Devoxx 2016

Devoxx france 2015 - Coding Fast and Slow

Devoxx 2015, Atelier Ionic - 09/04/2015

Lescastcodeurs Devoxx France 2015

Devoxx 2016 - L'odyssée du continuous delivery