[Back to Home Page]

www.RomanBlack.com

"BlackNet" serial network protocol
A system for unlimited number of devices to transmit/receive serial data on 1 wire.
Originally written May 2009, added more diagrams and finally put on the net 27 Apr 2011.

What is it?

This is a system for many devices to be attached to one serial bus wire, where any devices can send/receive at will with no conflicts. I designed this for use with my home automation controllers, but it has other uses like multiple sensors or multiple controlled devices.

All information on this "BlackNet" page can be considered open source and used for whatever, but please mention me if reproducing any of this work.


Hardware



The hardware side is very simple. One common ground. One "bus data" wire which is pulled up to 12v (or 13.8v) by a pull-up resistor. I ended up using a constant current pull-up of 150mA, so that I can use devices powered from the bus wire itself. This means I only need a 2-wire cable for long runs like temperature sensors on my solar panels etc, as they only draw a couple mA each they can use "ghost power" from the bus which saves running power cables to these sensors. Most of my devices have 12v power cabling too so the ghost power is just an option.

The BlackNet bus data line is both a transmit and receive line, for all devices.

Transmit. Each device can transmit by pulling the bus wire low to send one or more serial bytes. I used standard 8N1 serial, 1 start bit, 8 data bits and 1 stop bit. If no device is transmitting, the bus wire remains high at 12v.

Receive. Each device can receive any byte which is on the bus. So any device is capable of hearing what is said by any other device.


Desired protocol system

What I wanted from this system;
  • Only 2 wires needed; bus and ground
  • No limit to number of devices*
  • Easy serial using UART or PIC bitbanging
  • No special hardware needed, just a PIC and a transistor
  • No limit on transmission distance or baudrate*
  • Ghost power option
  • Any device can listen if it chooses
  • Any device can transmit, or choose NOT to transmit
  • Transmitting device can transmit any number of bytes
  • If a device fails it doesn't affect other devices*
  • Devices can be added or disconnected at any time with no issues
  • Multiple masters (transmitters) with no issues
  • Multiple slaves (receivers) with no issues
  • Devices do not need serial receive capability or UART to work on the bus
  • All devices transmit in sequence
  • A failsafe way of identifying all transmitting devices
  • If any devices transmit garbage bytes it does not affect other devices

    I should clarify some points;
    There are no limits in the protocol to number of devices although some limitations will occur because of line loading. It would still be reasonable to have hundreds of devices on the network if it has been set up for that.

    The protocol itself has no limit on distance. For long distance networks the devices can just use a lower baudrate.

    A device that fails will not affect the network provided that it does not short circuit. Any other device failure including; loss of power or open circuit or fail to transmit or garbage transmit will have no effect on other devices.

    No device is REQUIRED to transmit. This is important, it means any device can transit only if it chooses to, without affecting the chain of other devices sending/receiving. This is good for devices sending an infrequent alarm or sensors that only need to "report in" after long intervals. It also means that if any device fails or is disconnected there is no effect on the other devices. Obviously this is also good for devices that are in low-power mode most of the time.

    All devices CAN listen, but no device is REQUIRED to listen. This is a big strength of the protocol. It means that a device that sends data on the bus is not required to have a UART to receive bytes from other devices. All it needs is to monitor that the bus is free for X amount of time.

    Any device can be a transmitter or a receiver or combination of both. This is designed so that some devices might be light switches (transmit), some might be lights (receive) some might do both tasks. There can be many controllers on the bus controlling the one receiving slave or vice versa. The system is designed to be very open-ended.


    The protocol

    I won't claim any great originality here, over the years I have seen many systems for a number of devices on one serial wire so I just copied different elements from existing systems to combine together to create a very open ended and simple 1-wire network to suit my needs.

    It is basic and sequential ie a "round robin"; No device can transmit until device1 transmits. Then device2 can transmit IF it chooses, as many bytes as it chooses. Then device 3 can transmit IF it chooses, etc. Every device gets a chance to talk, then that "loop" is over.

    For my home automation I chose to make device1 the master clock, so every 1/2 second it transmits the full date/time etc. So devices can only transmit once per 1/2 second. The protocol does not require any specific loop timing, it is only required that every device has had its CHANCE to talk, then device1 talks again to start the loop all over again.

    Making sure all devices talk in turn! Each device knows its own number. After device1 finishes talking, the bus goes free again. After the bus has been free for a period of T*2 then device 2 is allowed to start talking (if it chooses). Then after the bus has been free for a time of T*3 then device3 is allowed to start talking. Then after the bus is free for a time of T*4 device4 is allowed to talk and so on.



    If any device chooses NOT to talk, then one time period T later the next device can talk so there is no penalty in bus time. This is very efficient for devices like light switches etc that only need to talk when someone presses the switch.

    Yes this can become slow with a very large number of devices IF they all need to talk at once, as the total bus loop time is an exponential function of number of talking devices;

    The total number of wait periods (T) for any number of devices (n) is roughly given by the formula;
    total_T = n * (n / 2)
    So for 10 devices the total T time needed will be 50 T.
    For 100 devices the total T time needed will be 5000 T.
    (However this is only true if ALL 100 devices talk in that cycle. In a best case cycle where none of those 100 devices talk the total cycle time is only 100 T.)

    As the time period T is only a few uS this is still fine for 100+ devices to all talk in a 0.5 second bus loop time. The protocol is based on minimal hardware and open-endedness, speed comes last. The open-ended nature of the protocol allows any or all devices to NOT talk so there is an option for very large numbers of devices on the bus if they don't all need to talk in every loop. Obviously, there is no time used up by devices that only listen!


    About time period T and when a device knows it's turn to talk

    Originally I was going to use a large time period for T that is longer than the total time for one serial byte, so that some high bits in a byte are not mistaken for a finished transmission.

    However it only requires ONE safe period this length after each transmission to ensure transmission has finished, then after that the period T can be shorter than a serial byte. A period T of 20uS is fine even with cheap PICs running from 4MHz xtal (1uS timing). The single safe period should be longer than a serial byte, for example can be 200uS if using 56kbaud serial.

    Procedure for a cheap transmitting device detecting time of 7T;
    1. first, wait for any device talking (bus low)
    2. wait for bus going low-high, then reset timer
    3. if bus goes low again go back to 2
    4. if timer reaches safeperiod + 7T, start transmission!


    Practical uses

    Obviously this is not a good system for devices that need to send lots of dense data. However it would be a good system where the devices need to be cheap as the devices do not need UART to be able to talk on the network. They only need to sense the TIME the bus is free, then transmit a data packet using bit-banged serial.

    This could be useful for a very large number of cheap sensors on a 2 wire bus, like solar panel sensors that only need to talk occasionally to signal that each panel is working, or signal a panel fail condition.

    It can be ideal to network cheap sensors within a building, like hundreds of light switches (or people sensors) where they only need to talk if a switch or sensor is activated. The same network can have thousands of receiving devices (like smart lights) that can just listen to the bus to see if they are commanded on or off.

    It might be useful for industrial or alarm sensors, especially as all devices are in parallel there is no problem attaching or detaching devices to/from the bus at any time while the bus is running.


    Practical implementation for my own network

    I decided to make device1 be a system clock. This transmits a time code of "year-month-day-HH-MM-SS" every 1/2 a second, and always starts a bus cycle every 1/2 a second.

    All devices afer device1 can know the time (if they receive bus data) and this allows for obvious time related features where devices can switch on/off at specific times without those devices needing to talk on the bus.

    Another benefit is that devices can talk only on specific cycles, so a device that only needs to talk every 30 seconds (like a room temperature sensor) can do so and uses very little bus time in total.

    Another simple trick I added later was to make every device transmit its device number as the first byte (assuming it is a talking device). This is not required by the protocol but seemed useful for a number of reasons.

    For hardware I used a constant current bus pullup of 150mA, to 12v. The 12v is unregulated battery voltage generally 11v to 13.8v range. I have not used ghost power in any devices so far, as I used telephone style cable and run these 4 wires;
    +12v unreg
    +9v reg
    Data bus (pullup +12v 150mA)
    Ground



    As the bus data is normally high (same polarity as a PIC USART), it can be connected directly into a PIC USART RX pin through a 47k resistor. This allows minimal hardware receive with only one resistor (uses the PIC internal port pin diode).



    Output to the bus is through a simple NPN transistor that is capable of pulling the 150mA bus line low. That means pretty much any cheap small-signal NPN like a BC337. If the serial output is bit-banged this only requires one resistor and transistor, and the bit banged serial from the PIC output pin must be inverted compared to a normal PIC USART TX pin.



    However if the output is from the PIC USART TX pin a second transistor is needed for data inversion. I figured that for minimal devices they can bit bang the output, and more complex devices can afford another 5 cent transistor. The benefit is that the RX does not need inversion, and the bus line is normally high allowing ghost power.


    Problems and limitations

    Obviously this network is for small data packets, a handful of bytes per device. Although any talking device can transmit any number of bytes generally they should be transmitting a small amount of data.

    As it is a "round robin" system (taking turns), devices need to know their own device number and have individual device numbers. This was fine for my needs and should be ok for permanent installations but could be a big limitation in some systems. It should be possible for a master to track all device numbers in use, and identify new devices and assign them a number although I didn't bother with auto numbering and just manually assigned numbers to each node I built.

    There is a problem if a device fails short circuit as they are parallel on the bus the entire bus goes down. However this type of fault applies to most types of network busses and is easy to trace. A device that fails open circuit will not affect the bus.

    There is a time problem if there are a lot of devices that must talk during one cycle. My home system with 0.5 seconds per cycle is limited to about 100 devices talking at once. However most devices do not need to talk EVERY cycle, and bus time can easily be distributed this way. There are plenty of applications that can have a large number of talking or listening devices without all the devices needing to talk at once.


    A big speed improvement

    Because I made the talking devices transmit their number, this allows a "fast mode" where device8 does not have to wait the typical 8T time before talking. If it listens and hears device7 has just finished talking then device8 can talk immediately (as it knows it is next).

    Like most of the protocol this "fast mode" is open ended and any device may use fast mode or not without the entire network needing to be structured this way. It would be most useful for higher numbered devices that might be clumped in groups (to talk in groups). Of course if there are only 20 or 30 must-talk devices on a network then fast mode is really not needed.


    Ghost power

    Because the bus line spends most of its time high (especially when there is a lot of devices!) it is ideal to provide ghost power to small low-power devices.

    The bus would charge caps in all devices and provide enough power to run the devices. With low power PICs and low quiescent current regulators common and cheap now it is conceivable for a simple 2-wire bus to power a large number of devices as well as carrying data.

    A practical ghost power circuit in a device would have a diode and resistor charging a large cap to approx the bus high voltage (12v). The resistor is needed to reduce peak currents after any 0 bit. As the data 0 and 1 bits are detected at logic levels (approx 1v and 3.5v) the data on the bus will not be affected by the ghost power charging caps etc which will ideally occur in the 9v to 12v range. The cap in each device should be large enough so that it's voltage does not drop much below 9v when the bus is transmitting data.

    Ideally a 5v (or 3.3v) low quiescent curent low dropout voltage regulator would be used. If so, a device might use less than 1mA average current from the bus so it would be possible to have a quite a lot of ghost powered devices on a network. Although I like the idea of ghost power I did not use it on my internal 4-wire home network. I may explore ghost power devices with an external 2-wire network at a later date.


    My home network

    I won't go into the exact details of my network setup as it performs some security tasks but here is a glimpse at the PCB I whipped up in 2009 for my BlackNet master clock (device1).



    At the right you can see the 4 wires to the network system, and in the circled area you can see that the serial output system just uses 2 SMD transistors and a few resistors.

    The master clock also has a multi-digit display and a Dallas Semi 32kHz temperature compensated precision oscillator module to give extremely accurate timekeeping. It is accurate to 2PPM which is around 1 minute a year error! I also have a GPS module that I will eventually connect as another node on the network that will act as a second "clock" and send error data to the master clock and allow the master clock to gently adjust it's time to re-synchronise to match the GPS clock.




    Minimum parts count - multiple devices

    (minor update 30th Apr 2011 - in response to a question I was asked about minimal hardware applications).

    For an extremely low parts count solution you can connect multiple microcontrollers on the one data bus line and use a 5v data bus. This uses a simple 270 ohm pull-up resistor for the entire data bus.

    This method can be used for multiple PICs on a single PCB or within a single device (like a large display unit).

    All that is needed is one diode per PIC microcontroller if you are using the USART TX output. If you are bit-banging serial output on low cost PICs (like a multiple sensor array) then the diode is not needed as the firmware can toggle the serial output between pull-down and high impedance states.

    This simple hardware will still allow hundreds of devices on the one data bus line, although the 5v line is not suitable for ghost power and will not generally allow the long distances obtainable from a 12v line and constant current pull-up.






    - end -

    [Back to Home Page]