how to connect applications/non-vpp-agent containers with ligato/dev-vpp-agent container ?

[zurrehma]

[ondrej-fabry]

Hey,

Take a look at this example: https://github.com/ligato/vpp-agent/blob/master/examples/localclient_linux/veth/main.go

Here is section which defines where the linux veth interface will be created:

Namespace: &linux_intf.LinuxInterfaces_Interface_Namespace{
	Type: linux_intf.LinuxInterfaces_Interface_Namespace_NAMED_NS,
	Name: "ns2",
},

https://github.com/ligato/vpp-agent/blob/master/examples/localclient_linux/veth/main.go#L282-L285

You can use type MICROSERVICE_REF_NS (defined here) which will create the linux veth interface inside container which has to have environmental variable MICROSERVICE_LABEL=xxx where xxx is name defined in the model next to type.

So for example:

Namespace: &linux_intf.LinuxInterfaces_Interface_Namespace{
	Type: linux_intf.LinuxInterfaces_Interface_Namespace_MICROSERVICE_REF_NS,
	Name: "myapp",
},

will create linux veth interface inside container with environmental variable MICROSERVICE_LABEL=myapp.

The container with the agent and VPP needs to have access to docker client. You can use flag -v /var/run/docker.sock:/var/run/docker.sock for the container to give it access. Note that you also need to use --privileged flag (maybe also --pid=host).

I hope this helps.

---

P.S: I was not able to reply to your email today, because we don't have proper IPv6 connection and your domain from which you sent the email only seem to support IPv6. 😁

[zurrehma]

uptil now i have only used ligato > vpp agent > vpp in container which i have pulled from docker hub and i have tested it with vpp-agent-ctl through which i have just created tap interface and now i want to connect my containers to the vpp which is running inside a container.

[lukasmacko]

not sure if I fully understand your desired topology. However, IMHO the best starting point for exploring contiv is to try vagrants ...using the provided script you'll get a VMs where contiv is already up and running. Give it a try

[zurrehma]

ok let me simplify my problem .i have pulled ligato/dev-vpp-agent image from docker hub, now let say i want to write new plugin and for writing new plugin i have to test my written code so my question is where and how i will test my plugin.

[ondrej-fabry]

You can test it in the image using Go unit tests. And for integration tests we use robot tests running on Jenkins. You can find the robot tests here: https://github.com/ligato/vpp-agent/tree/dev/tests/robot

[zurrehma]

@ondrej-fabry You can test it in the image using Go unit tests what this line means?

[zurrehma]

@ondrej-fabry **And for integration tests we use robot tests running on Jenkins.**also elaborate this if you have a time.

[ondrej-fabry]

now let say i want to write new plugin and for writing new plugin i have to test my written code

Maybe I'm not sure what you were going for. What kind of plugin are you building?

We test our software using unit tests which are contained in _test.go files and tested for each commit using Travis.

Then we also have integration tests written using robot framework and they are executed when something gets merged into pantheon-dev/master branch.

[zurrehma]

@ondrej-fabry for example there is a sample plugin in vpp known as macswap, now i want to provide its implementation in my local ligato/dev-vpp-agent container so what steps i will follow to do this?
of course if i am writing some plugin or some code i have to test it how i will test it locally?
for now just take me as a fresh in this field and i don't know more then what you guys have provided on this page https://github.com/ligato/vpp-agent/blob/master/docker/dev/README.md
so please tell me how from writing to testing a new plugin will be done.

[zurrehma]

any help about my above comment???

[ondrej-fabry]

Generating Go code for your VPP plugin

When you have VPP installed (via .deb packages), it will put it's API definitions in JSON format into /usr/share/vpp/api.

We use this directory to generate Go code for the binary API of VPP, located here: https://github.com/ligato/vpp-agent/tree/pantheon-dev/plugins/vpp/binapi
In the gen.go file you can find the commands run for generation.

All of this code generation for binary API is invoked via make target generate-binapi (in Makefile).

If you add your sample plugin to the list in gen.go file and run make generate-binapi it should generate Go code for you, which can later be used to call API in your VPP plugin.

Defining proto model for your VPP plugin

Inside plugins/vpp/model we have definitions for the various models we support currently, these represent the data model which is put into ETCD/GRPC.

You can create special directory for your data model and call make target generate-proto to generate the Go code from the proto model definition. Then you should probably also define key prefix for this model using keys.go file in the same package. Here's example for IPSec.

Creating configurator for your VPP plugin

I recommend creating special directory at plugins/vpp for your VPP plugin. Inside this directory, an implementation for your configurator would be located.

For calls to VPP we use vppcalls packages to create some easy to use methods which directly used the generated code, e.g. IPSec uses this vppcalls package.

The configurator is initialized via plugins/vpp, which dispatches changes and resyncs for the various configurators. This is where you would put else if ... cases to handle your own configurator.

---

Try to look around how we handle some specific configurator like IPSec to understand more.

[zurrehma]

@ondrej-fabry thanks now i have some entry point to start exploring the code.one other question is there any way to use this tap interface vpp-agent-ctl /opt/vpp-agent/dev/etcd.conf -tap which i have made inside my vpp router outside of this container so that it become visible in my host networking stack and then i connect my application containers to this tap interface.

[ondrej-fabry]

Do you mean like linux tap interface? Try using -ltap which creates linux tap interface.

[zurrehma]

@ondrej-fabry how you are defining proto model for any vpp plugin? means you are writing those definations inside https://github.com/ligato/vpp-agent/tree/pantheon-dev/plugins/vpp/model by yourself or it is also created the same way as we did make generate-binapi to generate go code.

second thing what this paragraph means:
You can create special directory for your data model and call make target generate-proto to generate the Go code from the proto model definition. Then you should probably also define key prefix for this model using keys.go file in the same package. Here's example for IPSec.

[ondrej-fabry]

You have to write the .proto file by yourself depending on what you will need, most of it will probably match fields from the binary API.

For example for IPSec when calling generate-proto it will call //go:generate directive from here.

[zurrehma]

@ondrej-fabry thanks, now i will look in to these protocol buffers and how to write them.one more thing before going for demo is that if i am writing these .proto files and creating these configurator for my sample plugin how i will test them that i am writing these files write? and other thing can i make some sort of topology to test this sample plugin after writing it completely with out errors in deployment via some tap interfaces or some other veth pairs ?

[ondrej-fabry]

You could use the vpp-agent-ctl program to add some sample data for your model.

[zurrehma]

@ondrej-fabry yeah i can use https://github.com/ligato/vpp-agent/tree/master/cmd/vpp-agent-ctl this but just like https://github.com/ligato/vpp-agent/tree/master/k8s/perf-demo these scenerios can i use vpp which is inside this ligato/dev-vpp-agent container to communicate with other containers e.g " let suppose i have two different containers container1 and container2 now i want to create layer2 bridge inside my vpp and then i want to somehow expose this layer2 bridge to my host and then finally want to connect my container1 and container2 to this bridge via some tap interfaces or veth. is this possible? " i am not sure that actually you got my point, if not then kindly ask me i will rephrase it.

[zurrehma]

@ondrej-fabry any help about my above comment or just see this https://wiki.fd.io/view/VPP/Configure_VPP_TAP_Interfaces_For_Container_Routing maybe then its easy to understand my above question.

[VladoLavor]

@zurrehma so basically you want to connect two containers using the VPP. Technically you have two options: use tap interfaces or combination of veth/af-packet. Also you need to understand namespaces. Namespace is an abstract environment, the vpp-agent supports several types but the most important are the named namespace and the microservice type namespace. The latter is more interesting for you, because it represents a container.

Tap: you need to configure tap interface in the VPP. You can use the vpp-agent-ctl for it with the comand -tap. When you verify that the interface was created in the VPP, check interfaces in the VPP host - you should see a linux tap-type interface (the default name is tap1, but it may differ). This is the "other end" of the VPP interface. What you want to do now is to apply a configuration on it and move it to the proper namespace (microservice type, which represents your container as I have mentioned above). The linux tap interface (see model) defines also a namespace - this is how you "move" the tap to a container. The vpp-agent-ctl command is -ltap.

Veth/Af-packet: second option is to configure a veth interface pair and attach an af-packet to the one on the host. A veth is linux interface (defined in the same proto file asa tap) which is always created in pairs. One end should be configured in the host (where the VPP resides) and the other end in the container you want to connect (again, use namspace for that). Now you need to add the VPP to the chain - you need to use the af-packet type interface. So create it in the VPP and attach it to the veth interface in the host (field 'host_if_name' in the model).

In any case, vpp-agent needs to know about containers, otherwise it won't be able to configure anything on them. You need to define environment variable MICROSERVICE_LABEL with some name, which will be used in configuration as namespace name.

Note: if you connect several containers and you want to ping between them, you need to put the VPP interfaces (taps, af-packets, whatever you choose) to bridge domain.

Also I can point you to examples for both scenarios, tap and veth - you can try it. Example configuration uses named namespaces instead of containers, so it can run on single host.

[zurrehma]

@VladoLavor when i create tap interface i can see it in my vpp but i am not understanding that where its other end is or where vpp host is ?

[VladoLavor]

@zurrehma the VPP host is vm/docker container where the VPP is running

[zurrehma]

@VladoLavor actually i have pulled only https://hub.docker.com/r/ligato/dev-vpp-agent/ this image. inside this image i have my vpp-agent and vpp and when i make tap interface using this command vpp-agent-ctl /opt/vpp-agent/dev/etcd.conf -tap i can see it in my vpp which is running in same container using show interfaces and after that i have no clue what to do and how to do?

[zurrehma]

@VladoLavor any help?

[VladoLavor]

@zurrehma :

i can see it in my vpp which is running in same container using show interfaces and after that i have no clue what to do and how to do?

You need to specify what you want to do next. Do you want to configure tap linux side?

[zurrehma]

Generating Go code for your VPP plugin

When you have VPP installed (via .deb packages), it will put it's API definitions in JSON format into /usr/share/vpp/api.

We use this directory to generate Go code for the binary API of VPP, located here: https://github.com/ligato/vpp-agent/tree/pantheon-dev/plugins/vpp/binapi
In the gen.go file you can find the commands run for generation.

All of this code generation for binary API is invoked via make target generate-binapi (in Makefile).

If you add your sample plugin to the list in gen.go file and run make generate-binapi it should generate Go code for you, which can later be used to call API in your VPP plugin.

Defining proto model for your VPP plugin

Inside plugins/vpp/model we have definitions for the various models we support currently, these represent the data model which is put into ETCD/GRPC.

You can create special directory for your data model and call make target generate-proto to generate the Go code from the proto model definition. Then you should probably also define key prefix for this model using keys.go file in the same package. Here's example for IPSec.

Creating configurator for your VPP plugin

I recommend creating special directory at plugins/vpp for your VPP plugin. Inside this directory, an implementation for your configurator would be located.

For calls to VPP we use vppcalls packages to create some easy to use methods which directly used the generated code, e.g. IPSec uses this vppcalls package.

The configurator is initialized via plugins/vpp, which dispatches changes and resyncs for the various configurators. This is where you would put else if ... cases to handle your own configurator.

Try to look around how we handle some specific configurator like IPSec to understand more.

@ondrej-fabry i have writeen my plugin configurator files and have initialized it with plugins/vpp, and have also put my if else cases here but when i run vpp-agent, it initialize my plugin but configuration from etcd is not coming?is there any file where i have not handeled my plugin?
when i run vpp-agent-ctl /opt/vpp-agent/dev/etcd.conf -acl it works fine.
vpp-agent-ctl /opt/vpp-agent/dev/etcd.conf -abf for my plugin is not working.
Note: i have not written any resync files for my plugin.

[ondrej-fabry]

Note that we already move to Agent v2 which has a lot of breaking changes] so it's gonna be little harder to support you.

Just from top of my head.. did you register your prefix here?

vpp-agent/plugins/vpp/data_resync.go

Lines 931 to 952 in 8c0fb64

	Watch("Config VPP default plug:IF/L2/L3", plugin.changeChan, plugin.resyncConfigChan,
	acl.Prefix,
	interfaces.Prefix,
	bfd.SessionPrefix,
	bfd.AuthKeysPrefix,
	bfd.EchoFunctionPrefix,
	l2.BdPrefix,
	l2.XConnectPrefix,
	l3.VrfPrefix,
	l3.ArpPrefix,
	l3.ProxyARPInterfacePrefix,
	l3.ProxyARPRangePrefix,
	l3.IPScanNeighPrefix,
	l4.FeaturesPrefix,
	l4.NamespacesPrefix,
	stn.Prefix,
	nat.GlobalPrefix,
	nat.SNatPrefix,
	nat.DNatPrefix,
	punt.Prefix,
	ipsec.KeyPrefix,
	srv6.BasePrefix(),