2
Kouji Matsui - kekyo• NAGOYA city, AICHI pref., JP
• Twitter – @kekyo2 / Facebook
• ux-spiral corporation
• Microsoft Most Valuable Professional VS and DevTech 2015-
• Certified Scrum master / Scrum product owner
• Center CLR organizer.
• .NET/C#/F#/IL/metaprogramming or like…
• Bike rider
CC-BY-SA 4.0 COPYRIGHT (C) 2016 KOUJI MATSUI
4
Agenda
• Introduction / Background
• How to build coreclr/corefx
• Add custom IL opcode
• Deep-dive CLR JIT
• Verify custom IL opcode to work
• Conclusion
CC-BY-SA 4.0 COPYRIGHT (C) 2016 KOUJI MATSUI
5
Introduction / Background
• .NET Core is open-sourced!!
• Become clearing the .NET internal implementations.
• .NET Framework noeq .NET Core, but very interesting internal implements anythings…
CC-BY-SA 4.0 COPYRIGHT (C) 2016 KOUJI MATSUI
6
Introduction / Background
• I am joining .NET Fringe Japan organizer teams. And thinking what about speaks first conference…
• Roslyn and corefx already exploring and explaining any people (in Japan) … Hmm.
• If can add custom IL opcode and build custom CLR ?Fun, interesting and understanding internal CoreCLR ! :)
CC-BY-SA 4.0 COPYRIGHT (C) 2016 KOUJI MATSUI
7
How to build coreclr/corefx
• Development and test bench requirements:• Windows 10 x64
• Visual Studio 2015 Update 3 (Using C++ compiler)
• CMake 3.6.2 (Multiplatform building tool) https://cmake.org/
• Python 3.5.2 https://www.python.org/
• Official docs: “Building and running tests on Windows”https://github.com/dotnet/coreclr/blob/master/Documentation/building/windows-test-instructions.md
CC-BY-SA 4.0 COPYRIGHT (C) 2016 KOUJI MATSUI
8
How to build coreclr/corefx
• Test summary:1. Get source codes from GitHub dotnet/coreclr, corefx.
• git clone https://github.com/dotnet/coreclr
• git clone https://github.com/dotnet/corefx
2. Build coreclr and corefx.
• Run build.cmd both coreclr and corefx.
CC-BY-SA 4.0 COPYRIGHT (C) 2016 KOUJI MATSUI
9
How to build coreclr/corefx
3. Test running minimum sample code using coreclr/corefx.
• Copy System.Runtime.dll and some assemblies from corefx into coreclr.
• Compile the C# Hello world code using VS2015 C# compiler (csc.exe), with /nostdlib /r:System.Runtime.dll and another strict options.
• Run the code, “CoreRun.exe Program.exe”
Need more informations?see documents previous links.
CC-BY-SA 4.0 COPYRIGHT (C) 2016 KOUJI MATSUI
10
How to build coreclr/corefx
• TIPS!!!• Must use stable version commits for coreclr
and corefx!
• They are developing continuously and worldwide, 10 or more commits/day.
• Hint: Look for CI status on GitHub coreclr/corefx page.https://github.com/dotnet/coreclrhttps://github.com/dotnet/corefx
CC-BY-SA 4.0 COPYRIGHT (C) 2016 KOUJI MATSUI
12
• This is just beginning :)
• Suggest first step: Very simple spec opcode:• Opcode name: “customcode”
• No operand, no IL stack consume/produce.
• Use opcode: affect output demonstration string to Windows Debug log.(Use Win32 API “OutputDebugStringW”)
Add custom IL opcode
CC-BY-SA 4.0 COPYRIGHT (C) 2016 KOUJI MATSUI
13
• Debug log can capture use Sysinternals DebugView utility.https://technet.microsoft.com/en-us/sysinternals/debugview.aspx
Add custom IL opcode
CC-BY-SA 4.0 COPYRIGHT (C) 2016 KOUJI MATSUI
14
Add custom IL opcode
• Thinking what currently declared opcode for completely nothing input/output and no side-effect in CLR ?
CC-BY-SA 4.0 COPYRIGHT (C) 2016 KOUJI MATSUI
15
Add custom IL opcode
• ex: Opcode “break” – Break execution the attached debugger current position.https://msdn.microsoft.com/en-us/library/system.reflection.emit.opcodes.break(v=vs.110).aspx
• “Debugger break” means raise interruption native CPU (x64), such as “DebugBreak” API or “__debugbreak” intrinsic.
• So, maybe contains invoke these API in coreclr source codes. I can find related code easier, try using base for this opcode…
CC-BY-SA 4.0 COPYRIGHT (C) 2016 KOUJI MATSUI
16
Add custom IL opcode• Grep special-like or unique naming opcode in coreclr:
ex: “initobj”, “ldftn” etc…--> Opcodes declared in src/inc/opcode.def by OPDEF() macro.
• Opcode break: “CEE_BREAK”OPDEF(CEE_BREAK, "break", Pop0, Push0, InlineNone, IPrimitive, 1,
0xFF, 0x01, BREAK)
• Add “CEE_CUSTOMCODE” for last opcode “CEE_UNUSED70”’s next:OPDEF(CEE_CUSTOMCODE, “customcode", Pop0, Push0, InlineNone,
IPrimitive, 2, 0xFE, 0x23, NEXT)
New 2 words opcode: fe,23Instruction move hint:
“NEXT” is execute next opcode.(Standard behavior)
No stack consume/produce
CC-BY-SA 4.0 COPYRIGHT (C) 2016 KOUJI MATSUI
17
Deep-dive CLR JIT
• Oh, I’m just declared new opcode “customcode” now!! :)
• But this opcode used no coreclr runtime…
Require giving new opcode behavior
MANUALLY
CC-BY-SA 4.0 COPYRIGHT (C) 2016 KOUJI MATSUI
18
Compiler-Importer JIT_CustomCode()JIT
Deep-dive CLR JIT
• How interpret opcodes in coreclr:
Assembly file:MSIL section Parse Call OutputDebugStringW()
Parse IL opcodes
GTNODE
Call
Peek from JIT helper function pointer table.
Internal IL stream tree structures
CC-BY-SA 4.0 COPYRIGHT (C) 2016 KOUJI MATSUI
19
Deep-dive CLR JIT
• Compiler-Importer (src/jit/importer.cpp) is IL opcode stream parser use declaring CEE_* macros.
• CEE_BREAK case example:case CEE_BREAK:
op1 = gtNewHelperCallNode(CORINFO_HELP_USER_BREAKPOINT, TYP_VOID);
goto SPILL_APPEND;
• “gtNewHelperCallNode” is construction GTNODE internal tree structure node for invoke JIT helper function.
• “CORINFO_HELP_USER_BREAKPOINT” is JIT helper function index symbol.
CC-BY-SA 4.0 COPYRIGHT (C) 2016 KOUJI MATSUI
20
Deep-dive CLR JIT
• “CORINFO_HELP_USER_BREAKPOINT” declared in src/inc/corinfo.h:• Symbol declared in “enum CorInfoHelpFunc”. Because JIT helper
functions management by function pointer table. This table size calculated from enum symbols count.
• And src/inc/jithelper.h:JITHELPER(CORINFO_HELP_USER_BREAKPOINT, JIT_UserBreakpoint,
CORINFO_HELP_SIG_REG_ONLY)
REAL helper function name
CC-BY-SA 4.0 COPYRIGHT (C) 2016 KOUJI MATSUI
21
Deep-dive CLR JIT
• Add “CORINFO_HELP_CUSTOMCODE” into CorInfoHelpFuncand provide JIT helper function information by JITHELPER() macro.JITHELPER(CORINFO_HELP_CUSTOMCODE, JIT_CustomCode,
CORINFO_HELP_SIG_REG_ONLY)
• Back to Importer and add “CEE_CUSTOMCODE” handler:case CEE_CUSTOMCODE:
op1 = gtNewHelperCallNode(CORINFO_HELP_CUSTOMCODE, TYP_VOID);
goto SPILL_APPEND;
• “TYP_VOID” is hold opcode value type (ex: operand type). “customcode” opcode hold no value, so this ID is TYP_VOID.
• Importer done!
New JIT helper function name
CC-BY-SA 4.0 COPYRIGHT (C) 2016 KOUJI MATSUI
22
Deep-dive CLR JIT• Implement JIT helper function “JIT_CustomCode”:HCIMPL0(void, JIT_CustomCode)
{
FCALL_CONTRACT;
HELPER_METHOD_FRAME_BEGIN_NOPOLL();
::OutputDebugStringW(L"Triggered custom code!!!!!!! (for JIT)");
HELPER_METHOD_FRAME_END_POLL();
}
HCIMPLEND
• HCIMPL0(), FCALL_CONTRACT, HELPER_METHOD_FRAME_BEGIN_NOPOLL(), HELPER_METHOD_FRAME_END_POLL() macros are required for construct hard-coded low level prologue/epilogue codes JIT helper functions.
• JIT helper function done!!
THIS IS CORE CODE!!
CC-BY-SA 4.0 COPYRIGHT (C) 2016 KOUJI MATSUI
23
Deep-dive CLR JIT
• Anything done ?• More two non-important points:
1. Implement interpreter-based code. Interpreter is src/vm/interpreter.cpp.
• But Windows-x64 environments nothing to use interpreter, all situation works only use JIT.
2. ILFormatter (src/utilcode/ilformatter.cpp).• Format printer-friendly string from IL opcode. But default
implementation is printing uses CEE_* macro information, this session’s custom code is not required.
CC-BY-SA 4.0 COPYRIGHT (C) 2016 KOUJI MATSUI
25
Verify custom IL opcode to work
• OK, ready to run… How?• The “customcode” IL opcode can work with CLR now, but how to generate
“customcode” contained assembly??
Manually paching with BINARY EDITOR…??
(; ゚Д゚)
This is TOP SECRET:I fogot IMAGE_DOS_HEADER, IMAGE_FILE_HEADER, IMAGE_NT_HEADER,
IMAGE_OPTIONAL_HEADER, IMAGE_DATA_DIRECTORY, IMAGE…
CC-BY-SA 4.0 COPYRIGHT (C) 2016 KOUJI MATSUI
26
Verify custom IL opcode to work
• Thinking about more easy construction:1. Compile standard C# sample code by .NET Core 1.0.
2. Use “ildasm” to disassembled.
3. Insert “customcode” opcode into disassembled IL source code.
4. Use custom-opcode enabled “idasm” to build new assembly.
• The “ilasm” and “ildasm” are built with coreclr. New opcodes can handling from “opcode.def” automatically.
CC-BY-SA 4.0 COPYRIGHT (C) 2016 KOUJI MATSUI
27
Verify custom IL opcode to work• Bootstrap test code in C#:
Generate template code from “dotnet new” command and simplied:namespace ConsoleApplication{public static class Program{public static void Main(string[] args){}
}}
• Compile:• dotnet restore• dotnet build• Storing compiled assembly: bin¥Debug¥netcoreapp1.0¥addil.dll
CC-BY-SA 4.0 COPYRIGHT (C) 2016 KOUJI MATSUI
28
Verify custom IL opcode to work
• Disassembling:• ..¥ildasm.exe bin¥Debug¥netcoreapp1.0¥addil.dll > addil.il
• Fixed attributes for referenced System.Runtime:.assembly extern System.Runtime
{
.publickeytoken = (B0 3F 5F 7F 11 D5 0A 3A )
.ver 4:2:0:0
}
Fix pubkey token and version similer to your local built corefx binaries if different.
CC-BY-SA 4.0 COPYRIGHT (C) 2016 KOUJI MATSUI
29
Verify custom IL opcode to work
• Insert “customcode” opcode into Main method body:.method public hidebysig static void Main(string[] args) cil managed
{
.entrypoint
.maxstack 8
IL_0000: nop
customcode
IL_0001: ret
}
Insert “customcode” opcode!!
CC-BY-SA 4.0 COPYRIGHT (C) 2016 KOUJI MATSUI
30
Verify custom IL opcode to work
• Assemble by customcode-enabled “ilasm”:• ..¥ilasm.exe Program.il
Success with nothing error.If use official ilasm.exe, will cause error:
“syntax error at token ‘ret’”
CC-BY-SA 4.0 COPYRIGHT (C) 2016 KOUJI MATSUI
31
Verify custom IL opcode to work
• Run the assembly and verify with DebugView:• Before execute DbgView.exe
• ..¥CoreRun.exe Program.exe
CC-BY-SA 4.0 COPYRIGHT (C) 2016 KOUJI MATSUI
33
Conclusion
• Custom IL opcode declare and implement:
• Declare opcode into opcode.def with OPDEF() macro.
• Declare JIT helper function into corinfo.h and jithelper.h with JITHELPER() macro.
• Implement JIT helper function with HCIMPL() macros.
• Implement custom opcode handler into Compile-Importer.
CC-BY-SA 4.0 COPYRIGHT (C) 2016 KOUJI MATSUI
34
Conclusion
• Verification:
• Generate IL codes from disassembled C# codes with “ildasm.”(Or, write from scratch IL codes…)
• Using custom-opcode enabled “ilasm” to generate final binary.
CC-BY-SA 4.0 COPYRIGHT (C) 2016 KOUJI MATSUI
35
Conclusion
• coreclr is truly OSS: Custom IL opcode can implements with average difficulity.• This session explain with constraints “No operand, No stack
consume/produce opcode.”Maybe more hard work for intermediate usage opcode design…
• But YOU CAN DO THAT!!
• This session’s demonstration code:
• https://github.com/kekyo/coreclr git branch:addil
CC-BY-SA 4.0 COPYRIGHT (C) 2016 KOUJI MATSUI
36
Thank you joining my session!!
• Become slides on slideshare and my blog entry.http://www.kekyo.net/ (Sorry blog only Japanese language)
• My twitter account @kekyo2, follow <3
• GitHub https://github.com/kekyo/ contains:• FusionTasks, RelaxVersioner, fscx and more…
• Open conference with “Center CLR” in Aichi pref., Japan! Join us!!https://www.meetup.com/en-US/CenterCLR/
CC-BY-SA 4.0 COPYRIGHT (C) 2016 KOUJI MATSUI